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Executive Summary 

Headquartered in Washington, DC, Urban Alliance serves at-risk youth through its high school 
internship program, which provides training, mentoring, and work experience to high school seniors 
from distressed communities in Washington, DC; Baltimore, MD; Northern Virginia; and Chicago, IL 
The program serves youth before they become disconnected from school and work, with the goal of 
helping them successfully transition to higher education or employment after graduation. 

Urban Alliance has commissioned the Urban Institute to conduct a six-year, randomized controlled 
trial impact and process evaluation of its high school internship program. A previous report (Theodos et 
al. 2014) provided a process analysis of the program and baseline information about Urban Alliance and 
the youth participating in its high school internship program in Washington, DC, and Baltimore in the 
2011-12 and 2012-13 program years. This report continues that research by presenting early 
adulthood impacts of the Urban Alliance program on these youth, including college attendance and job 
preparation. It will be followed by a final impact report expected in 2017. Below is a summary of the 
present report. 

What Is the Urban Alliance Program Model? 

■ Targeting: Urban Alliance targets seniors in high school at risk of not transitioning to further 
education or meaningful work. It aims to serve "middle of the road” high school students who 
maintain a grade point average (GPA) of 2.0 to 3.0, but it is not limited to that group. 
Participants must have enough course credits accumulated to allow for an early-release 
schedule during much of senior year. 

■ Program components: The program's key elements are (1) a paid internship in an office setting at 
a nonprofit organization, corporation, or government agency (Monday through Thursday after 
school and full-time during the summer); (2) soft and hard skills job training for four to six 
weeks after school at the start of the program ("pre-work training”), followed by ongoing 
training after school each Friday ("workshops”); (3) coaching and mentoring provided by Urban 
Alliance program coordinators and job mentors at the internship site; and (4) alumni services 
consisting of individual coaching, alumni reunions and events, and a paid internship opportunity 
during the summer break from college. 


EXECUTIVE SUMMARY 


VII 



What Was the Study Design? 


■ Random assignment: We assigned 2011-12 and 2012-13 program applicants in Washington, 
DC, and Baltimore at random to a treatment or control group. We randomized applicants 
separately at each site at a two-to-one ratio of treatment to control to maintain the minimum 
number of enrolled youths required by the program. Youth enrolled in the study, together with 
their parents, gave consent for researchers to collect their program, high school, and 
postsecondary school data, as well as to be contacted to complete a survey. Urban Alliance 
invited those assigned to the treatment group to participate in the program. Those applicants 
assigned to the control group were not invited to participate and thus had access to neither the 
job training nor the internship. 

■ Data co//ect;on: We collected quantitative data from a numberof sources, described below. 

» Application baseline data: Urban Alliance gave applicants a 12-page application form that 
requested detailed contact information, demographics, GPA, attendance record (to be 
completed by school counselors), goals, career interests, work history, household structure, 
and one teacher and one nonteacher recommendation. 

» School-level high school baseline data: We collected aggregated 2011 school-level 

performance data from boards of education in Maryland and Washington, DC, for each 
school youth attended in those jurisdictions and data from the National Center for 
Education Statistics 2010 Common Core for school-level information about free and 
reduced-price lunch eligibility and racial composition. 

» Neighborhood baseline data: We accessed characteristics of neighborhoods including 

unemployment rates, poverty rates, and racial and ethnic composition from the American 
Community Survey 2008-12. 

» Youth-level high school baseline and outcome data: With Urban Alliance’s help, we obtained 
individual-level data— including CPA, attendance, and graduation— from the DC and 
Baltimore public school systems, the DC Public Charter School Board, and individual 
charter schools. 

» Program implementation data: We used data provided by Urban Alliance on case 

management status, participation in program activities, and biweekly wages paid to interns. 
» Interim survey outcome data: We collected survey data from the control and treatment 
groups of both cohorts about their educational, employment, and well-being outcomes; 
their high school experiences; and their postsecondary education preparation. The survey 
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was fielded 9 to 14 months after youths’ predicted high school graduation date via website, 
telephone, and an in-person interviewer. It achieved a 77 percent response rate. 

» Postsecondary institution outcome data: We used data from the National Student 

Clearinghouse, which provides information on college enrollment for most colleges in the 
United States. We also used National Center for Education Statistics’ Integrated 
Postsecondary Education Data System data to assess the quality of institutions that youth 
attended. 

■ Anaiysis methods: Our randomized controlled trial approach allowed us to estimate the causal 
impacts of the Urban Alliance program by measuring differences in outcomes between the 
treatment and control groups. Below are further details on our analytic approach. 

» Predicting treatment take-up: Before considering program impacts, we investigated which 

characteristics predicted youth participation at three key points in the program. We used 
regression analysis to estimate how much individual and neighborhood characteristics 
contributed to the probability of attending pre-work, completing pre-work, and completing 
the internship. 

» Intent to treat analysis: We used the intent to treat method to estimate impacts. In this 
model we compared the average outcomes of those youth offered access to the program 
(treatment) to the outcomes of those denied access (control). The causal effects of access 
to the program are simply the differences in outcomes between the two groups. Because 
sampling variation can lead to differences in the average characteristics of treatment and 
control participants, which may then influence youth outcomes, we used regression 
analysis to control for these measured differences. We estimated separate regressions for 
each GPA group (low, middle, high), each site, and each gender. For each of these groups 
there was variation in the potential for changes in outcomes. 

» Treatment on the treated: We also used treatment on the treated analysis, which allowed us 
to estimate the effects of completing the program, that is, effects that may have been 
drowned out in the intent to treat because of the high levels of program attrition. Here we 
looked at the differences in each outcome between the treated (youth who completed the 
program) and control subjects. To correct for the fact that persistence in the program 
required information about certain characteristics we were unable to measure, we 
estimated the treatment on the treated effects by using an instrumental variables 
approach. 
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Who Were the Youth in the Study? 


■ The study sample included 1,062 youths who applied to the Urban Alliance program and agreed 
to participate in the evaluation. Below is an overview of their baseline characteristics. 

» Demographic characteristics: The average age for all applicants at the start of the program 
was 17. Ninety percent of applicants were non-Hispanic African American (“African 
American" for shorthand), and 65 percent of applicants were female. Over half lived only 
with their mother, 5 percent only with their father, one-quarter lived in two-parent homes, 
and 12 percent lived with neither parent. Four percent had children of their own. 

» Work experience: Three-quarters of youth reported in the baseline survey at least some 
work experience before applying for the program, with average experience of just less than 
10 months in all jobs combined. Common job experience was through summer jobs, 
including those accessed through the Summer Youth Employment Program in Washington, 
DC. 

» Educationai background: Slightly more than a quarter of Urban Alliance applicants attended 
a charter school, with the majority in Washington, DC. Applicants on average exhibited 
satisfactory but not stellar school performance. The average cumulative GPA at the end of 
junior year was 2.7. Over one-third of applicants had attended more than one high school. 

» Neighborhood characteristics: Applicants typically resided in economically distressed 

neighborhoods. More than three-quarters lived in a neighborhood with an unemployment 
rate greater than 10 percent, and nearly half lived in neighborhoods with poverty rates 
higher than 25 percent. Almost all applicants resided in census tracts that were over 75 
percent minority. 

» Schooi characteristics: Almost all schools that Urban Alliance applicants attended were 

majority African American, though some schools in DC also had significant Hispanic 
student contingents. About 93 percent of youth attended schools with the majority of 
students eligible for free or reduced-price school lunch. Forty-five percent of Urban 
Alliance applicants attended a school ranking in the bottom quartile of proficiency in 
reading and math in DC or Maryland. 

■ Differences between treatment and controi groups: There were very few differences across the 
treatment and control groups at baseline, indicating that randomization was successful for this 
study. However, a higher proportion of the control group (97 versus 95 percent) reported being 
US citizens and had a banking account (42 versus 35 percent). In addition, the treatment group 
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on average had a higher GPAthan the control group (2.7 versus 2.6) at the end of their junior 
years. 

■ Differences across sites: Across sites, there were a number of differences in applicants. The 
majority of students at each site were African American, but there was a higher percentage of 
white applicants in Baltimore and a higher percentage of Hispanic applicants in DC. There were 
more English language learners in DC than Baltimore. Youth in DC reported having saved less 
money than Baltimore youth. 

What Services Did Youth Receive? 

The Urban Alliance model allows youth to self-select into the program and expects, by design, varying 
levels of attrition, depending on the site. A student could self-select out of the program in three ways: 
by not showing up to pre-work training, not completing the pre-work training, or not completing his or 
her internship. 

■ Program attrition: There is substantial attrition in the Urban Alliance internship program, 
primarily in the first two stages, before and during pre-work training. Of those students 
assigned to the treatment group, 22 percent did not attend any pre-work sessions. It is worth 
noting that although these youth were part of the study— and as a result, their attrition matters 
in terms of measuring programmatic impacts— Urban Alliance does not consider those who 
applied for the program, but did not show up to pre-work training, as a program exit. Of all 
treatment group youth, one-quarter (25 percent) began but did not complete pre-work 
training. The remaining 52 percent were placed in a job and most of those (84 percent) 
completed the program. In all, 41 percent of treatment group youth completed the program. 

■ Take-up regressions: We estimated predictive models that relate baseline characteristics of the 
youth and program to the likelihood that youth will complete each of the program stages. 

» Predictors of attending pre-work training: Youth who previously held a job had a probability of 
attending pre-work roughly 9 percentage points higher than their counterparts. Taking 
special education courses and being a parent were both negatively associated with 
attending pre-work. Program year 2011-12 was associated with a higher rate of showing 
up to pre-work than program year 2012- 13. 

» Predictors of compieting pre-work, conditionai on attending pre-work: Youth in DC were less 
likely to complete pre-work relative to Baltimore youth. The probability of completing pre- 
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work training was 26 to 28 percentage points higher for those students with GPAs of 2.0 to 
4.0 than for those with GPAs below 2.0. 

» Predictors of completing an internship, conditional on completing pre-work: DC youth were 
more likely to successfully complete their internships. Additionally, a reduction in the 
caseload of a youth’s program coordinator by 10 youths decreased the probability of 
attrition at the internship stage by 9 percentage points. 

» Predictors of completing an internship, unconditional on previous steps: Completion did not 
vary by site, gender, parenting, employment history, family structure, language spoken at 
home, neighborhood poverty, or taking special education courses. The probability of 
completing an internship was 19 to 23 percentage points higher for those students with 
GPAs of 2.0 to 4.0 than for those with GPAs below 2.0. Additionally, youth from the 2011- 
12 cohort were 14 percentage points more likely to complete the program than youth from 
the 2012-13 cohort. 

■ Reasons for attrition: Our process evaluation revealed many reasons why youth exited the 
program. Youth principally cited competing priorities, such as athletics or extracurricular 
activities, class schedules, lack of interest in the training, family or personal issues, relocation, 
and cost of transportation as reasons for attrition. During the internship, only a small 
proportion of youth exited the program. Some youth were fired from their internship, primarily 
for poor attendance; however, some youth exited for positive reasons, such as to pursue other 
educational or job opportunities. 

■ Services received: We estimated whether access to the Urban Alliance program led more 
treatment group youth to receive educational and job services. It may seem definitional that 
the program led to increased service take-up, but many youth already receive college and 
career preparatory services in their high school or through after-school programs. (We have no 
quality measures for these other high school or after-school programs.) We found that the 
difference between the treatment and control group of receiving college help and job help was 
statistically significant, but relatively small, at 8 percentage points. For those students who 
completed the Urban Alliance program, the difference was roughly double this. Receiving 
college and job help was quite prevalent among youth in the control group, more than 80 
percent of whom reported accessing this form of support. In comparison, more than 90 percent 
of the youth in the treatment group reported receiving job or college help. 
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What Impacts Did the Program Generate? 


■ Education and employment preparation: We estimated the impacts of Urban Alliance on 
measures of college and job preparation. We found positive and significant impacts of the 
program along several dimensions of education and employment preparation measures. 

» Education preparation: We estimated the impact of Urban Alliance on the probability of 
youth taking the SAT, taking the ACT, and filling out the Free Application for Federal 
Student Aid (FAFSA); their average comfort with completing the FAFSA and applying for 
scholarships; and their probability of applying to college. The program had positive and 
significant impacts on youth taking the ACT and on their comfort with completing the 
FAFSA and other scholarship applications. However, we found no impact of the program on 
the probability of taking the SAT or filling out the FAFSA. This finding is likely because most 
youth complete these precollege tasks anyway; for example, in the control group 92 
percent of youth filled out the FAFSA, 91 percent took the SAT, and 92 percent applied to 
college. 

» Job preparation: To test youths' job preparation, we estimated the impact of the program on 
average job application comfort, hard skill comfort, and soft skill comfort. We found that 
Urban Alliance had a positive and significant impact on both hard skill and soft skill comfort. 
This result is notable, as research literature is increasingly showing the importance of soft 
skills for youths’ academic and employment success. In addition, DC also showed positive 
impacts in job application comfort, though the combined DC and Baltimore model did not. 

■ Educational attainment 

» High school: We estimated the impacts of Urban Alliance on high school educational 
outcomes including the probability of high school graduation, suspension senior year, 
chronic absence senior year, and cumulative GPA senior year. We found no significant 
impacts for the overall sample. However, we did find that males in the treatment group 
were 4 percentage points more likely to graduate high school than males in the control 
group. Males were less likely to have received job or college help in high school than 
females, and males also had lower rates of achievement in high school. Thus, Urban Alliance 
likely had a greater opportunity to make a difference for males than females. 

» College: We estimated the extent to which Urban Alliance affected college attendance and 
the quality of the college youth attended. For the full sample, we did not detect impacts on 
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college attendance measures. Although 74 percent of youth completing the Urban Alliance 
program ("treated youth") and 66 percent of those offered access to the Urban Alliance 
program ("treatment youth") attended college compared with 60 percent of youth in the 
control group, the differences were not statistically significant in the regression analysis. 
Urban Alliance did have a positive and significant impact on the quality of the school 
attended as measured by the 75th percentile SAT score. 

» College for males: Although twice as many females as males applied to participate in the 
Urban Alliance program, we did not find any effects on college attendance or quality for 
females. However, the program demonstrated large impacts on males in the probability of 
attending college and the probability of attending a four-year college. Males offered access 
to the Urban Alliance program were 11 percentage points more likely to attend college, and 
those completing the program were 26 percentage points more likely to attend college 
than those in the control group. This result was driven by males in the treatment group 
attendingfour-year colleges rather than two-year colleges. It appears that the Urban 
Alliance program benefits males by boosting their rate of college attendance to the rate 
females reach even without the program. 

» College for middle-tier students: Urban Alliance had no significant impact on college 
attendance or quality for the low-GPA subgroup; college enrollment was low for that 
group. However, we did see notable gains for students in the middle-GPA (2.0 to below 3.0) 
subgroup. Indeed, this is the group the Urban Alliance program was initially designed to 
serve, although the program does not restrict admission to those with lower or higher 
GPAs. For the middle GPA subgroup, youth offered treatment and those completing the 
internship were both more likely to attend a four-year college than the control group (by 
12 and 21 percentage points, respectively). Interestingly, the likelihood of attending a two- 
year college went down for the treatment and treated groups relative to the control group. 
In this way, it appears that the Urban Alliance program helped middle-performing students 
who otherwise might have attended two-year colleges to instead attend four-year colleges. 
The program did not demonstrate any college attendance or quality impacts for high- 
performing youth, that is, those with GPAs 3.0 or greater. It appears that college 
attendance was already fairly prevalent for this group. 

■ Employment, wages, and savings: In addition to exploring the effect of Urban Alliance on 
educational outcomes, we also estimated the impacts on employment, wages, and savings. 
Measuring these outcomes for youth requires a longer time horizon because most of these 
youth are still enrolled in some sort of postsecondary institution. At the point of the interim 
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survey, we did not find significant effects on the probability of having a job, wages, or money 
accumulated. However, we did find that Urban Alliance led to a decrease in the probability of 
having a job for males. This finding likely indicates that Urban Alliance encouraged males to 
attend college and therefore away from working after graduating high school. 


EXECUTIVE SUMMARY 


XV 




Introduction 


Effective programs to help disadvantaged youth become self-sufficient, attend college, and embark on a 
path toward career success are critical. Urban Alliance, headquartered in Washington, DC, serves such 
youth through its high school internship program, which provides high school seniors at risk of 
disconnecting from work and school with training, mentoring, and work experience, with the goal of 
helping them successfully transition to higher education or employment after graduation. Youth 
growing up in low-income and low-opportunity communities, such as those targeted by Urban Alliance, 
face formidable challenges in transitioning to adulthood despite some recent efforts to revitalize 
disinvested neighborhoods and reform struggling school systems. Schools in high-poverty areas often 
lack sufficient resources and offer inadequate instruction; moreover, because of family, neighborhood, 
and peer environment factors, low-income children attending these schools have difficulty taking 
advantage of the educational opportunities that do exist (Jacob and Ludwig 2009). By attending college 
and acquiring job skills, youth in these communities increase their chances of future economic stability, 
but they are less likely to do so than their more advantaged peers. 

In Washington, DC, where the cost of living is high, over a quarter (26.5 percent) of children under 
age 18 live below the federal poverty level. In Baltimore, MD, the share of children in poverty is even 
higher, at 34.1 percent.^ Many of the schools in these cities have poor academic outcomes. In both 
Baltimore and DC public schools, around two-thirds (69 percent and 58 percent, respectively) of 
students who enter ninth grade graduate within four years.^ The students who make it to graduation 
are often unprepared for life after high school. Many high school seniors in both cities’ public school 
systems are not proficient in core subjects such as math and English. Unsurprisingly, many of DC and 
Baltimore's young residents do not attend college, have limited options for future skill development, 
and face unemployment. 

Since its founding in 1996, Urban Alliance has placed over 2,000 youths in internships, growing to 
serve over 500 interns annually in four sites: Baltimore (since 2008), Chicago (2012), Northern Virginia 
(2013), and its original site, Washington, DC. As part of this expansion process. Urban Alliance has 
commissioned the Urban Institute to conduct a six-year, randomized controlled trial (RCT) impact and 
process evaluation of its high school internship program. 

This report describes the early adulthood impacts of the Urban Alliance program, including college 
attendance and job preparation. The report is part of a larger impact and process evaluation of the 
Urban Alliance high school internship program. Fora detailed description of the implementation of the 
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Urban Alliance internship program, see Theodos et al. (2014). We will assess longer-term impacts in a 
report expected in 2017. 
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Background 


Barriers to Education 

Despite rising overall rates of college attendance in recent decades, many youth from disadvantaged 
backgrounds still do not enroll in or complete any postsecondary education. Compared to white, non- 
Hispanic youth ages 18 to 21, of whom 52 percent were enrolled in college, only 40 percent of African 
American youth and 40 percent of Hispanic youth were enrolled in college in 2013.^ Eighty percent of 
recent high school completers from high-income families enroll in college, compared with 49 percent of 
completers from low-income families.'^ 

These trends also apply to college completion. Of first-time students starting at four-year 
institutions in 2007, 63 percent of white students graduated within six years, compared with only 41 
percent of African American students and 53 percent of Hispanic students.^ The discrepancies are even 
more striking by income. Of financially dependent youth with at least some postsecondary education, 
96 percent of those in the highest family income quartile earned a bachelor’s degree by age 24, but only 
22 percent of those in the lowest quartile did (Pell Institute 2015). 

The lower rates of college entrance and completion for disadvantaged youth have staggering 
consequences for their future careers, lifetime earnings, and economic stability. Median earnings of 
adults age 25 and up with a high school diploma were $27,528; those with a bachelor’s degree earned 
$50,254 (in 2013 dollars).^ Over a lifetime, a person with a bachelor’s degree will earn about two-thirds 
more over her working life than a high school graduate.^ 

Many factors lead to reduced college access and success for disadvantaged youth. These youth, 
some of whom would be the first in their families to attend college, often lack the support and guidance 
in their homes and communities that are necessary to prepare for and apply to college. One study found 
the largest predictor of college success to be the intensity and quality of high school curricula (Adelman 
1999), which are often lacking in high schools in disadvantaged neighborhoods. These factors are even 
higher predictors for students of color than for white students. A follow-up study also found high school 
academic intensity to be the most important predictor (Adelman 2006). Later studies have found that 
students enrolled in remedial education courses in college are less likely to earn a degree (Wirt et al. 
2004) and that taking a rigorous course increases the number of college credits earned and college 
grade point average (GPA) for students enrolled in four-year colleges (Long, Conger, and latarola 2012). 
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other barriers to college are financial: the rising cost of attendance, limited federal financial aid, 
and insufficient financial resources. Average tuition and fees at private ($31,231) or public ($9,139) 
four-year colleges in 2014-15 were more than triple the cost 30 years prior in real dollars (College 
Board 2014). Though there are more federal financial aid programs now than in the past, and they are 
often larger and serve more types of students (Dynarski and Scott-Clayton 2013), aid has not kept pace 
with rising costs and there have been recent cuts, such as reductions in Pell grants in 2011. Moreover, 
youth— particularly African American, Latino, and low-income youth— often lack awareness of college 
costs and financial aid options (George-Jackson 2015). Among students who leave college after one 
year or less, 31 percent cite financial reasons (Ross et al. 2012). 

Other research suggests that the level of expectation for college attendance in a community 
influences rates of college entrance (Derden and Miller 2014), that contacts with high school 
counselors regarding information on college are associated with applying to college (Bryan et al. 2011), 
and that more frequent parent-youth discussions about education-related issues are associated with 
greater odds of enrolling in a four-year college, though the benefit of these discussions is smaller for 
African American youth than for other youth (Perna and Titus 2005). Youth who rely heavily on peers, 
rather than parents or school personnel, for information on the college transition are less likely to apply 
to selective colleges (Hill, Bregman, and Andrade 2015). 

Even youth who do not enter college still need support in preparing for a career. Youth ages 16 to 
19 suffer high unemployment (20 percent in 2014), especially African American (33 percent) and 
Hispanic (23 percent) youth.® Whether from a lack of work experience or of job skills training, youth are 
often not ready for entry-level jobs. In one study, 49 percent of youth who graduated high school but 
did enter college felt that high school left them unprepared for the work habits they would need in the 
workforce, and employers estimated that 39 percent of recent high school graduates were unprepared 
for their jobs (Achieve, Inc. 2005). Though career success is heavily dependent on educational 
attainment, lack of career preparation may also be a barrier to career success for disadvantaged youth, 
especially those who will not complete a four-year college degree. 

Programs to Promote Success 

Many programs exist to help prepare youth for college and career. Programs focus on many types of 
youth, including high school students, dropouts, and youth who have experience in the juvenile justice 
or child welfare systems. Some programs are coordinate within high schools and offer workplace skills 
and experience as a part of secondary education, sometimes through internships inside or outside of the 
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school. Examples are career academies— partially self-contained occupationally themed subschools 
within high schools— and magnet schools. Other programs, like Urban Alliance, are run by private 
organizations or social service departments, rather than through collaborations with the education 
system. These programs may offer internships, job skills training, or both. Some programs focus on 
college readiness, aiming to help students graduate high school and enroll in college prepared for the 
challenge; they may offer instruction, tutoring, and academic counseling. Another approach is to offer 
general case management and/or mentoring, with links to other supportive education and employment 
services as needed. Finally, some programs take a holistic approach and offer a combination of job 
training or internship and academic or college preparation counseling, perhaps with other types of 
supports or general mentoring as well. 

Evidence from Youth Programs 

A sizable literature exists describing the impacts of these different types of programs designed to 
improve the educational and career outcomes of youth. Although no studies have rigorously evaluated 
a program with Urban Alliance’s unique combination of work experience, training, mentorship and 
coaching, and alumni support, studies have assessed programs offering different combinations of these 
supports. Several studies have not yielded evidence of positive long-term outcomes. However, many 
studies have only tracked outcomes in the shortterm, and the major federal evaluations of youth 
employment programs have focused on programs geared toward disconnected youth rather than youth 
still in traditional high school settings. Much can be learned from programs such as Urban Alliance, 
which provides a comprehensive and intensive array of services to students who are still in high school 
and are in danger of becoming disconnected from education or employment. 


Work Experience 

A review of research on the effect of work experience on youth academic and career outcomes, outside 
of any structured program, shows there may be a positive relationship between employment during 
high school and later outcomes. Some nonexperimental longitudinal studies have shown that holding a 
job during high school is associated with higher academic success. Light (1999) found that students with 
jobs during high school who worked a moderate number of hours per week (less than 20) performed 
better in school than students who did not work at all. Ruhm (1995) found that students working 20 
hours per week had significantly higher earnings six to nine years later than their peers who did not 
work during high school. Rothstein (2001) found positive relationships between teenage employment 
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and future employment and education: teens with a moderate level of work at age 16 and 17 worked 
about six more weeks per year at age 18 to 30 than those who had not worked as teens. Furthermore, 
teens who worked up to 20 hours per week were more likely to have at least some college education by 
age 30. Conversely, later studies have found no positive correlation between teen and later adult 
employment (Rothstein 2007; Tyler 2003). 


Work-Based Learning 

Secondary education programs that connect students to internships, combine learning with a job, or in 
some other way provide youth with an on-the-job learning experience can prove beneficial. In an RCT of 
career academies, Kemple (2008) found that participants experienced higher levels of interpersonal 
support from peers and teachers, and those students who entered school at high risk of dropping out 
were more likely to stay through 12th grade. Eight years after entering the program, participants had 
earnings and employment higher than nonacademy students in their high schools. A quasi-experimental 
study using school administrative data and surveys found that students in career academies were 9 
percent more likely to graduate and 12 percent more likely to attend a postsecondary institution than 
students in general and vocational tracks (Maxwell and Rubin 1997). Similarly, studies of career magnet 
schools, which specialize in one particular career theme (such as agricultural science or business), have 
found that they result in lower dropout rates and increased student investment in school (Katz et al. 
1995). Findings have been mixed on whether they improve academic achievement (Ballou, Goldring, 
and Liu 2006; Cobb, Bifulco, and Bell 2009). In comparing the quality of school-based employment with 
outside employment, one nonexperimental study found that students report school-based jobs are 
lower in quality, but these jobs may offer important work experiences to youth who would have 
difficulty finding work on their own (Hamilton and Sumner 2012). 

Some work-based learning programs operating outside schools have been shown to increase the 
academic performance and classroom attendance of participating students while decreasing delinquent 
behaviors outside class. One study found a positive effect on test scores for youth who participated in a 
local government internship compared to a control group (Hamilton and Zeldin 1987). An RCT 
evaluation of New York City’s Center for Economic Opportunity youth literacy program found that 
students with a paid summer internship to complement their literacy, math, and job skills education 
attended more class hours and improved their math grade a full letter grade more than those without 
the internship (NYC Center for Economic Opportunity 2011). A summer youth employment program in 
Boston was found to reduce adverse social behaviors, such as violence and drug use among participants. 


6 


EMBARKING ON COLLEGE AND CAREER 



compared to those in a comparison group consisting of the program's waiting list (Sum,Trubskyy, and 
McHugh 2013). A Chicago program offering high school students paid summer jobs and a job mentor 
found that youth randomly assigned to be placed in a job had nearly 4 fewer violent-crime arrests per 
100 youth than youth assigned to a control group that were not placed in a job (Heller 2014). A random 
assignment evaluation of Youth Corps, a federally funded program providing paid jobs for youth age 18 
to 24, with academic support for those needing a GED (General Educational Development) 
certification,^ found no impacts on educational attainment or employment in an 18-month follow-up 
survey. However, participants were 7 percentage points more likely to report that they planned to 
complete at least some college (Price et al. 2011). 

There are few evaluations of programs combining an internship with other academic or social 
supports. An RCT evaluation of After School Matters, which offers high school students paid 
apprenticeship-type experiences in many settings, found no impacts on marketable job skills or 
academic outcomes, but did find a reduction in problem behaviors and more markers of positive youth 
development among the treatment group (Hirsch et al. 2011). 

The Summer Career Exploration Program in Philadelphia, which provides high school students with 
a summer job in the private sector, preemployment training, and a college student mentor, was found in 
an RCT to have no effect on students’ high school graduation, college enrollment, attitudes toward work 
or school, or sense of self-efficacy. The program’s only positive impact was that participants were more 
likely to enroll in a college preparatory or specialized academic program (12 percent, compared with 8 
percent for the control group; see McClanahan, Sipe, and Smith 2004). 

It is unclear whether programs like the Summer Career Exploration Program (which lasted only for 
the summer) would be more effective if they were longer term. A quasi-experimental study of a Boston 
school-to-career initiative offering youth intensive academic instruction, worksite learning experiences, 
and post-high school supports found positive impacts for youth who participated in the program 
compared with a control group of youth who would have met the program’s eligibility standards had 
they applied. It found that the program group members were 6 percentage points more likely to attend 
college, with an even more pronounced positive effect for African Americans (Jobs for the Future 
1998). 

An RCT of Year Up, a year-long program that combines job training, a paid internship, mentoring 
and counseling, and job search or college application assistance, found a positive impact on earnings 
three years after program completion. However, participants were less likely than youth in the control 
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group to be attending college three years after completion, though among those in college, participants 
were more likely to attend full-time and receive financial aid (Roder and Elliot 2014). 


JobTraining 

Programs that offeryouth job training without direct job experience have documented some success. 
Participation in Job Corps, a federally funded program providing vocational training, academic support, 
counseling, and often residential living, was found in an RCT to have short-term impacts on earnings, 
employment, education, and crime. However, after 5 to 10 years these impacts disappeared for the 
sample as a whole, which contained youth ages 16 to 24 at the time of application, with the impact on 
earnings remaining significant only for the subgroup of youth ages 20 to 24 (Schochet, Burghardt, and 
McConnell 2006). Additionally, an impact study of the JobTraining Partnership Act, a previous federal 
program, found positive impacts on adult earnings and employment but little or no effect on youth 
employment or the earnings of female youth; it found a negative impact on the earnings of male youth 
(Bloom et al. 1993). An RCT evaluation of replications of San Jose’s Center for Employment T raining, a 
training program for high school dropouts, found no lasting impact of the program on earnings or 
employment (Miller et al. 2005), but the authors note this finding may have been caused by widespread 
infidelity to the Center for Employment Training program model. Two studies of the program in the 
early 1990s found positive impacts on employment and earnings (Burghardt et al. 1992; Cave et al. 
1993). Perhaps the most promising recent job training program evaluation is that of the National Guard 
Youth ChalleNGe program. This program, which provides short-term job and life skills training in a 
quasi-military environment and includes follow-up mentoring, demonstrated long-term positive effects 
on employment: after three years, the randomly assigned program group had an employment rate 7 
percentage points higher and earnings 20 percent higher than a comparison group, and program 
participants were more likely to obtain college credits or a high school diploma or GED (Millenky et al. 
2011). 

Case Management and Mentoring 

Case management and mentoring programs have documented generally positive results, at least in the 
near and medium term. Impact studies of the Big Brothers Big Sisters mentoring program have reported 
mixed findings. One study found that treatment group members skipped half as many days of school, 
had slightly better GPAs, and had an improved concept of their scholastic competence (Tierney, 


8 


EMBARKING ON COLLEGE AND CAREER 



Grossman, and Resch 1995). Another study also found improved academic confidence and 
performance, but only at first; impacts disappeared by 15 months (Herrera et al. 2011). Grossman and 
Rhodes (2002) found that youth enrolled in the program for more than 12 months had significant gains 
at 18 months in self-worth, perceived scholastic competence, relationships with parents, and other 
positive social outcomes. A specialized Big Brothers Big Sisters program for children of incarcerated 
parents found treatment group youth had higher self-esteem and felt more connected to school, 
community, and family at 18 months, but they did not differ in their academic competence or attitudes 
(US Department of Justice 2011). A Philadelphia-area program that provides mentoring for all four 
years of high school found that students offered a mentor had college attendance rates in the first two 
years after high school graduation that were 20 percentage points higher than those of their peers 
(Johnson 1999). 


College Access and Readiness 

College access and readiness programs have had mixed results as well. Upward Bound, a federally 
funded program lasting up to four years and offering instruction, tutoring, and counseling, was found to 
have no overall impact on high school graduation or college enrollment. However, it was found to 
improve education outcomes for students with initially low educational expectations. These students 
were more than twice as likely (38 versus 18 percent) to enroll at four-year colleges than similar control 
group members (Myers et al. 2004). The random assignment evaluation of the Quantum Opportunities 
Project, which operated in five sites across the country and offered case management, academic 
support, developmental activities, and community service, found no positive impacts, though this 
finding was attributed to poor implementation of the program model and low participation (Schirm, 
Stuart, and McKie 2006). Harvill and colleagues’ (2012) meta-analysis of 14 college-access program 
experimental or quasi-experimental evaluations found an average boost to high school graduation rates 
of 8 percentage points. However, when only the three experimental evaluations are considered, the 
impact is not statistically significant. The analysis also found an average increase in college enrollment 
of 12 percentage points, whether all evaluations or only the experimental ones are considered. 

In all, there is evidence to suggest that programs offering underserved youth jobs, job training, 
career-focused education, mentoring, or college readiness activities— or some combination of these— 
may be effective in helping youth achieve better outcomes. However, we know little about the effects of 
intensive initiatives for students still in high school that provide not only a paid job, but also 
individualized support and continual training. From the existing evidence, it is difficult to determine if 
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the sort of outcomes achieved by the National Guard Youth ChalleNGe program— that is, positive 
impacts on earnings and employment— could be achieved by a program that takes place during the 
school year and does not include a residential component. A rigorous evaluation of the Urban Alliance 
internship program will help us know more. 
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Urban Alliance High School 
Internship Program Model 

Urban Alliance has developed a program model designed to address the organization’s goals of 
empowering underresourced youth to aspire, work, and succeed through paid internships, formal 
training, and mentoring. Urban Alliance targets its internship program to a subset of high school-age 
youth reached through a months-long recruitment process. 

Target Population and Recruitment 

Urban Alliance targets students at selected schools (further described in the Study Participants section) 
that they consider to have a high proportion of youth at risk of not connecting to further education or 
meaningful work. The organization seeks out youth who will be in their senior year of high school during 
the upcoming program year. These youth will need to have enough course credits accumulated to allow 
for an early-release schedule during the internship phase of the program. The Urban Alliance program is 
targeted to high school seniors because program staff believe the program is most effective at reaching 
young people during this transitional year; its lessons and curriculum are designed for youth about to 
enter adulthood. In addition, the program targets youth in their senior year because only by that point 
will they have accumulated enough credits to have a shortened school day schedule. 

The program aims to serve "middle of the road" high school students who maintain a GPA of 2.0 to 
3.0, but it is not limited to that group. Although Urban Alliance leadership believes students with GPAs 
that are too low will generally have insufficient time, resources, and course credits to participate in the 
program and graduate on time, the program often accepts youth with lower GPAs. The program also 
does not exclude youth with high GPAs, though these students often cannot participate in the program 
because, although they may have sufficient credits for an early-release schedule, they are more likely to 
be taking honors and Advanced Placement courses to fill up their schedules. Youth with high grades 
may also have higher skill levels and more external support, so their need for the program may be lower. 

Urban Alliance begins to recruit students for its high school internship program in the spring of 
students’ junior year, and recruitment continues into the fall of their senior year. The recruitment 
process differs between cities. In Washington, DC, the organization’s relationship with schools is 
informal. Urban Alliance presents its program during assemblies or in classrooms to high school juniors 
at several public and charter schools in the city. In Baltimore, the relationship with the school system is 
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formalized, and youth receive course credit for participating in the program. School counselors and 
teachers identify and refer students in their schools who they think will benefit most from the program. 
Many of these youth do not formally apply until they start pre-work training. In the more recently 
opened Chicago site, and also in the Northern Virginia site. Urban Alliance has also established a formal 
partnership with the local school districts, and youth receive course credit for participating in the 
program. 

Program Components 

The Urban Alliance high school internship program has four primary components: professional and life 
skills training, paid internships, coaching and mentoring, and alumni services. 


Training: Pre-work and Workshop 

Urban Alliance conducts training workshops from late September or early October of each school year 
through the end of July. This training includes three to six weeks (varying by city) of "pre-work” training 
before the start of the internship. Program participants are expected to attend training for one to one 
and a half hours everyday after school during that period. The primary goal of pre-work training is to 
prepare the youth for their internships. Topics include workplace etiquette and culture, as well as such 
hard skills as faxing and Microsoft Excel basics. Urban Alliance also uses these sessions to familiarize 
youth with post-high school education and employment options, financial literacy, and select life skills. 
Sometimes, youth are assigned homework. During pre-work training, youth receive training on job 
interviewing, which they then use in interviews with mentors at their prospective job sites. 

After the internships start in the late fall, youth are expected to attend workshops every Friday 
after school. Workshops focus heavily on topics related to post-high school planning, financial self- 
sufficiency and life skills, though they also continue to review workplace-relevant topics. After the 
school year ends, youth attend half-day workshops every Friday. 

Urban Alliance staff also prepare youth for a final presentation that interns give in July at Urban 
Alliance's public speaking challenge event. These are PowerPoint presentations designed by the youth 
to describe their recent internship experiences and career goals. A volunteer panel of community 
stakeholders judges the youth, who can receive a $100 prize for performance. Youth can also receive 
bonuses earlier in the year for participating at other events or participating in program activities while 
waiting on a delayed job placement. 
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Paid Internships 


Urban Alliance program staff pair students who complete pre-work training with paid internships based 
on each student’s skill levels, needs, interests, and the range of internships available. Starting in the late 
fall. Urban Alliance participants go to their internships from 2:00 to 5:00 p.m. after school Monday 
through Thursday. This schedule requires that interns obtain permission for an early-release class 
schedule during their senior year of high school. During the summer following graduation, Urban 
Alliance interns work full days Monday through Thursday. Urban Alliance partners with professional 
clothing nonprofits such as Dress for Success to give interns access to clothing. 

The settings and responsibilities for internships vary, but most are office settings and include such 
tasks as filing, copying, and answering phones. Other types of jobs include greeting and directing guests 
in hotels or banks. Some interns also work in educational or day care settings. Interns earn a starting 
hourly wage close to their city’s minimum wage ($8.25 in DC^° and $7.25 in Baltimore during this study), 
which can rise to $10.00 based on job performance and effort, including workshop attendance and 
communication with their assigned program coordinator. For the most part, interns are officially 
employed and paid by Urban Alliance while working at their internship sites, though select job sites pay 
interns directly. 


Coaching and Mentorship 

Youth receive job mentoring and general coaching as part of the program. In addition to running the 
training workshops described above, front-line staff (program coordinators) maintain coaching 
relationships with each youth assigned to their workshop group. Each program coordinator has a 
caseload of approximately 30 to 35 interns whom they support throughout the program. Coordinators 
track individual student performance in various areas including workshop and job attendance, 
punctuality, workshop homework assignments, academic progress, post-high school planning, and 
progress toward the presentation at the public speaking challenge. Program coordinators also send out 
a weekly e-mail to youth, and youth must check in with program coordinators at least once during the 
week. If interns are going to be late to work or miss work, they must contact their program coordinators 
and their employers. 

The program coordinators meet with each intern three times per year in a one-on-one meeting to 
discuss post-high school planning. They also provide ad hoc support; speak with youth before or after 
workshop sessions; discuss youths’ experiences in a group during a workshop; and keep in touch via 
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individual phone calls, e-mails, and texts. Some youth face serious challenges such as teen pregnancy, 
domestic or relationship abuse, problems with their home life, or housing instability. Program 
coordinators support youth emotionally and connect them with external resources to meet their needs. 

Each intern is also assigned to a "job mentor" or supervisor, who is an employee at the intern's 
workplace responsible for ensuring the intern has adequate and appropriate work, teaching the intern 
necessary skills, and, ideally, providing opportunities for enrichment and networking within the 
workplace. Job mentors assess interns' performance in the workplace. They may suggest possible 
termination of an intern's position if his or her attendance or performance is poor, but the program 
endeavors to resolve all performance issues except the most severe (e.g., time-sheet fraud). When 
performance concerns arise. Urban Alliance staff first establish a work contract with the youth. Only if 
poor performance persists after several intervention attempts will the organization fire the youth and 
ask the intern to leave the program. 


Alumni Services 

As the Urban Alliance program has grown. Urban Alliance program coordinators have increasingly 
found themselves providing informal support to youth who keep in touch after graduating from the 
program. In 2007, Urban Alliance first began offering informal education and career support services to 
alumni. More recently, it formalized this program component, adding regional alumni services directors. 
Through alumni services. Urban Alliance aims to prevent program alumni who are college students from 
dropping out and to link alumni with work. Alumni services also provide an avenue for tracking student 
outcomes after program completion. 

Services for alumni include ad hoc individual coaching meetings with youth, a resource room where 
alumni can access job search and education materials, networking opportunities through a website, 
alumni reunions, and connections to paid internship opportunities. 


Logic Model 

Initially, Urban Alliance measured its success sporadically and informally (Winkler, Gross, and Theodos 
2009), but the organization developed a formal logic model that details how it expects program 
activities to lead to specific outputs and, ultimately, outcomes for the youth served (figure l).The logic 
model describes both the four main activities that youth engage in (left-hand column) and a set of 
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outputs and related targets (middle column) associated with them. For example, the first activity is to 
place students in professional, paid internships to expose them to the world of work, and one target is 
that 70 percent of students invited to pre-work training complete it and are placed at a jobsite. Other 
outputs relate to the development of work skills and initiation of post-high school planning. As the logic 
model demonstrates, the majority of expected short- and long-term outcomes (right-hand column) 
relate to postsecondary education. Program staff articulate they hope most youth will first complete 
college before becoming employed; they also assert that the employment-readiness training is valuable 
for those youth who elect to enter the labor force rather than attend college or a technical or training 
program. 

The logic model highlights the short-term, intermediate, and long-term outcomes and indicators 
associated with the various program activities. In its early years. Urban Alliance developed five goals for 
participants in its high school internship program. Specifically, youth would (1) improve their hard and 
soft job skills, (2) gain long-term employment experience, (3) graduate from high school, (4) attend 
college or a training program, and (5) identify long-term employment opportunities. 

A final note about the program design relates to its funding: the internship program is financed 
directly by internship sites and philanthropic foundations. Approximately 75 percent of internship 
placement sites, most typically for-profit businesses but also nonprofit and governmental organizations, 
make a donation to Urban Alliance for each intern they hire; in DC the expected amount is $12,500, and 
in Baltimore it is $10,000. This donation is tax-deductiblefor the for-profit firms. Urban Alliance uses 
this contribution to pay intern wages and to cover the costs of services provided to the interns. It raises 
additional funds to cover the cost of placing students at job sites that cannot afford the $12,500 
donation. Under the current model, a city must have approximately 70 internship slots to be sustainable 
financially. 
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FIGURE 1 


Urban Alliance High School Internship Program Logic Model 


Urban Alliance High School Internship Program Logic Model 



URBAN< 


Activities Program Outputs 


1) Skill Training 

Interns receive training in professional soft 
skills, financial literacy and post-secondary 
planning before internship placement 
through Pre-Work and after internship 
placement during Friday Workshop. 

2) Direct Work Experience 

Interns are placed in internships to practice 
their skills and gain access and exposure to 
professional networks. 

3) Mentorship 

Interns are paired with jobsite mentors who 
supervise and provide feedback to work 
tasks, help establish professional networks, 
and advise on professional development and 
growth. 

4) Case Management 

Program Coordinators guide interns' 
transition Into work, liaise between the 
mentor and Intern, oversee post-high school 
planning, and resolve development issues as 
needed. 

5) Alumni Services 

Alumni Services teams continue to provide 
support to both disconnected and connected 
alumni 


Skills Training 

• 90% of interns receive 30 hours of pre-work 
training. 

• Regions maintain 75% workshop attendance rate. 

Internship Experience 

• 75% of interns complete the program. 

• 75% of interns achieve 500 hours of internship 
experience. 

• Interns work on average 80% of hours budgeted 
each month. 

• 80% of interns receive two or more skill set 
evaluations. 

Mentorship 

• 75% of interns report high mentor engagement in 
quarterly surveys. 

Case Management 

• PCs produce weekly tracking notes on each intern. 

• 95% of alumni have completed Post-High School 
Plan, including FAFSA, personal statement, resume, 
and college applications. 

Alumni Services 

• ASD contacts 70% of NSC alumni call list. 

• ASD provides services for 75% of confirmed 
disconnected alumni. 


Outcomes 


Short-Term Program Outcomes 

• 75% of interns show growth on 
professional skills assessment. 

• 85% of alumni immediately connect 
to a sufficiency pathway (college, 
career training, living-wage 
employment) 

Post-Program and Long-Term 

Outcomes 

• 80% of alumni are "Connected" one 
year post program 

• 75% of alumni who enroll in college 
persist into their second year 

• 50% of disconnected alumni are 
reconnected by alumni service. 

• 70% of alumni who enter 4*year 
college graduate within 6 years of 
entry. 

• 70% of alumni who enter 2-year 
colleges graduate within 3 years of 
entry. 


6) Employer Support 

Urban Alliance Program Coordinators support 
employers and jobsite mentors, ensuring 
both a rewardingand impactful internship 
experience. 


Employer Outputs 

• Employers report increased diversity, improved 
culture, and staff management training 
opportunities. 


Employer ROI Outcomes 
• 80% jobsite retention rate 


Source: Urban Alliance. 

Notes: NSC = National Student Clearinghouse; PC = program coordinators; FAFSA = Free Application for Federal Student Aid; 
ASD = alumni services department; ROI = return on investment. Outputs and outcomes for “interns” are targets among interns 
placed at job sites and those for "alumni” are targets among interns that complete an internship. 
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Evaluation Design 

This section details this study's approach to random assignment, data collection, and analysis methods. 
A detailed discussion of the survey methodology is located in appendix A. 

The approach was designed to answer three research questions: (1) Does the Urban Alliance 
program lead to increased rates of high school graduation and college enrollment for participants 
compared to a control group of youth not enrolled in the program? (2) Are youth who participate in the 
program more prepared for further education and employment (e.g., through greater job skills) than 
control group youth? (3) Do Urban Alliance participants have higher rates of employment and earnings 
than control group youth? 

Random Assignment 

For the purpose of this study, we assigned 2011-12 and 2012-13 program applicants in Washington, 
DC, and Baltimore at random to a treatment or control group. The decision to combine two program 
years was necessitated by insufficient sample size for impact measurement in any single-year cohort. 
The analyses focus solely on the Baltimore and Washington, DC, sites, because the organization’s 
expansion to Chicago in 2012 and Northern Virginia in 2013 occurred after the evaluation had begun. 

We randomized applicants separately at each site at a two-to-one ratio of treatment to control. 
(The ratio was dictated by Urban Alliance needing a sufficient number of youth to enroll in the 
internship program.) As part of the application, students gave researchers permission to collect their 
program, high school, and postsecondary school data as well as permission to be contacted to complete 
a survey. Consent to participate in the study was not a requirement to receive Urban Alliance services. 

Urban Alliance invited those assigned to the treatment group to participate in the program, which 
began with mandatory pre-work training before assignment to an internship position. Urban Alliance 
did not invite youth in the control group to participate in the training or internship. 

Data Collection 

Researchers collected quantitative data from a number of sources summarized below. 
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Application Baseline Data 


Urban Alliance high school internship program staff gave all applicants a 12-page application form to 
complete. The application requested detailed contact information, demographics, GPA, attendance 
record (to be completed by school counselors), goals, career interests, work history, household 
structure, one teacher and one nonteacher recommendation, and parental consent forms. The 
application also served as the baseline survey for the evaluation. Urban Alliance provided the Urban 
Institute with the paper applications for all applicants. We entered a subset of fields relevant for the 
study into an electronic database. Unfortunately, item nonresponse was high for some fields in the 
application, making it impossible to reliably analyze household income, receipt of public benefits, 
education level of household members, and recommenders’ assessments of youths’ hard and soft skills. 


Aggregated High School Baseline Data 

We also collected aggregated data about the high schools youth attended. We pulled school-level 
performance data for each school for 2011 from Maryland and Washington, DC’s boards of education.^^ 
To understand the relative performance of schools attended by Urban Alliance youth, information was 
gathered not only on the schools youth attended, but also on all schools in Maryland and DC according 
to their average 10th grade reading and math standardized test scores, determining each school’s 
percentile rank among schools in that state or district. 

Additionally, we used data from the National Center for Education Statistics 2010 Common Core 
for school-level information about free and reduced-price lunch eligibility and racial composition.^^ We 
linked the Common Core and performance data with Urban Alliance applicant records to better 
understand youths’ educational environments, opportunities, and challenges. 


Neighborhood Baseline Data 

The American Community Survey provided characteristics of neighborhoods including unemployment 
rates, poverty rates, and racial and ethnic composition. We matched study participant addresses to 
census tracts and accompanying indicators from the American Community Survey, 2008-12. 
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Youth-Level High School Baseline and Outcome Data 


With the help of Urban Alliance, we accessed student-level data from the DC and Baltimore public 
school systems, the DC Public Charter School Board, and individual charter schools. Data gathered 
included GPA, attendance, and other indicators such as whether students were in a special education 
program and whether they graduated high school. When data on GPA were missing, we used counselor- 
reported or student-reported GPAs from program applications. 


Program Implementation Data 

Urban Alliance tracked service delivery data on case management status and participation in program 
activities, noting youth attendance, progress in completing post-high school planning actions such as 
submitting applications for financial aid, and other important indicators. We relied on these indicators 
to define treatment status, intensity, and inform program activities. Urban Alliance also collected 
financial records of biweekly wages paid to interns. We used these records to calculate total earnings 
and to refine how long youth remained employed at their internships. 


Interim Outcome Survey Data 

We collected survey data from the control and treatment groups of both study cohorts about their 
educational, employment, and well-being outcomes. The survey also asked participants about their high 
school experiences, postsecondary education preparation, and family members' educational 
attainment. A subcontractor, SSRS, fielded the survey 9 to 14 months after youths’ predicted high 
school graduation date, capturing outcomes mostly during the first summer after high school 
graduation. Youth who completed the survey received a $40 gift card for their participation. SSRS 
implemented the survey online and via telephone. We augmented these efforts with an in-person 
interviewer. The survey achieved a 77 percent response rate overall with response rates for the 
treatment and control group at 80 and 75 percent, respectively. See appendix A for more detailed 
information on the survey methodology and response rates. Appendix B shows baseline characteristics 
for the analysis sample, providing an assessment of differential attrition across the treatment and 
control groups. 
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Postsecondary Institution Outcome Data 


The National Student Clearinghouse (NSC) provides information on college enrollment for most 
colleges in the United States, including data on date of enrollment and completion of each semester at 
the individual level for each institution attended. 

The analysis also used National Center for Education Statistics’ Integrated Postsecondary 
Education Data System (IPEDS) data. IPEDS provides data on postsecondary institutions, such as 
location, admissions rate, two- and four-year graduation rates, mean standardized admissions test 
scores, retention rate, and net attendance cost. We used these data to assess the quality of institutions 
that youth attended according to the NSC data. The measures of quality we chose were colleges’ 75th 
percentile for SAT scores, retention rates, and graduation rates. 

Analysis Methods 

This study used an RCT approach that allowed us to estimate the causal impacts that the Urban Alliance 
internship program had on youths’ education and employment outcomes. The impact of the internship 
program can be measured by differences in outcomes between the treatment and control groups. In 
what follows, we first describe our analytic technique for predicting treatment take-up. We next 
describe our approach to intent to treat (ITT) analyses and treatment on the treated (TOT) analyses. 

We did not include in our analysis variables with a high number of missing observations, such as 
whether youth had a checking or savings account. Among the variables used, only a small number of 
responses were missing, so we did not use imputation methods for missing data. Instead, we omitted the 
few observations with missing data from each analysis. 


Predicting T reatment Take-Up 

Throughout the Urban Alliance program year there were high levels of attrition for both cohorts. It is 
therefore important to first consider which characteristics predict youth participation before 
considering the impact of the internship program. To do so, we considered participation levels at three 
key points in the program: beginning of pre-work, completion of pre-work, and completion of the 
internship. To determine predictive characteristics, we first looked at baseline descriptive 
characteristics of the treatment group for those who reached each benchmark compared with those 
who did not, conditional on meeting the prior benchmark. 
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Using regression analysis, we estimated the probability of completing each stage of the program for 
those assigned to the treatment group controlling for a number of individual, school, and neighborhood 
characteristics. A logistic model was used with the following underlying variable structure: 

y,* = /So +/3iindi + /32neighi + e,- 

where ind, is a vector of individual characteristics from the application data and high school data 
including program year, gender, previous job experience, whether the student was a parent, family 
structure, site, language of the parent, special education enrollment, and GPA; neigh, is the percent of 
people in poverty in youths’ neighborhoods; and y, is the latent propensity that an individual will reach 
the benchmark. We do not observe y, directly, but rather we observe a binary variable for whether the 
individual reached a benchmark: 

|1 if 

These estimates tell us how much each characteristic contributes to the probability of attending 
pre-work, completing pre-work, and completing the internship. 

Urban Alliance recruited study participants from 38 high schools in DC and Baltimore. Students 
from the same high schools likely had related probabilities of attending the program or achieving 
certain outcomes such as attending college. Because students were not randomly assigned within high 
schools, the share of treatment students varied across high schools. This variation cannot be perfectly 
controlled for using school-level characteristics. For this reason, we used a random effects model to 
control for unobserved heterogeneity across different high schools in the probability of the outcomes.^^ 


Intent to T reat Analysis (ITT) 

To estimate the impact of Urban Alliance on youths' education and employment outcomes, we first used 
the ITT method, which analyzes outcomes based on initial assignment to either treatment or control 
groups. Not all youth in the treatment group completed the internship program, thus the term intent to 
treat. 

By basing analysis off of the exogenous assignment of youth, we can be certain that any effect 
found through this method is causal. This procedure reveals the effect of offering the program to 
interested students including those who take up the program but drop out and youth who do not show 
up at all. 
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Because we randomly assigned the students to the treatment and control groups, the differences in 
education and employment outcomes between the two groups are the causal effects of access to the 
program. We calculated these differences using data from the outcome survey and from the NSC. 

To account for sampling variation, we also used a regression-based approach to control for any 
measured differences between treatment and control groups. By including control variables we were 
able to reduce the amount of unexplained variance and sample size needed to detect an effect. Because 
control variables must be unaffected by access to treatment, all control variables used in our analysis 
were measured prior to randomization. Control variables included those that were statistically 
different between the two groups and had low item nonresponse rates. We used the following control 
variables in our regressions: program year, gender, percentage of youth’s neighborhood living in 
poverty, held a job prior to random assignment (Y/N), and junior year cumulative GPA. 

Impacts may vary across subgroups of youth. In particular, we believe the program may have had 
different effects on students with different levels of high school performance, at the two sites, and for 
males and females. For instance, as we show below, the rate of attending a four-year college for youth 
with GPAs of 3.0 to 4.0 in the control group was 72 percent. This result implies a limited potential to 
induce high-achieving students to go to college. In contrast, the rate of attending a four-year college for 
youth with GPAs of 2.0 to 3.0 in the control group was only 46 percent. Thus the potential of the 
program to affect an outcome like attending a four-year college may depend on GPA. To examine these 
effects, we divided the sample into three groups based on junior year cumulative GPA: less than 2.0, 2.0 
to 3.0, and 3.0 to 4.0. We also explored whether outcomes differed for youth in Washington, DC, and 
Baltimore and for males and females. We estimated separate regressions for each GPA group (low, 
middle, high), each site, and each gender. 

Using a procedure developed by Benjamin! and Hochberg (1995), we made adjustments for having 
conducted multiple tests of significance. Our tables show unadjusted significance levels, but we note in 
the text where and how the adjusted results differ from the unadjusted results. 


T reatment on the T reated (TOT) 

A different approach to estimate programmatic impacts, TOT, estimates the effects of completing the 
program rather than just the effects of access to treatment. Youth who completed the Urban Alliance 
program (i.e., they remained in the program until June 1) are considered "treated youth,” and those 
offered access to the Urban Alliance program are "treatment youth.” 
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The TOT method allowed us to estimate effects that may have been drowned out because of the 
high levels of attrition from the program. However, it has potential downsides because youth who 
completed the program may have been systematically different from those who did not complete the 
program. Because of these suspected systematic differences, TOT results are likely only internally valid 
to the group that completed the program. High levels of attrition suggest that persistence in the 
program requires certain levels of motivation, accommodating schedules, and many other unobservable 
factors. These factors may also influence the youth’s decision to pursue secondary education or 
employment. There is likely a similar group within the control group that would have been more likely to 
complete the program, and ideally, we would compare the outcomes of youth most likely to complete 
the program in the control group to those who did complete the program. Unfortunately, because of the 
unobservable nature of what influences youth to remain in the program, we cannot conduct this 
comparison. 

To correct for this selection bias, we estimated TOT effects by using an instrumental variables 
approach proposed by Angrist, Imbens, and Rubin (1996). For this estimate, known as the complier 
average causal effect, randomization into the treatment group is used as an instrument for actual 
treatment to remove some of the bias caused by selection into take-up. This methodology assumes a 
constant causal effect. 

The complier average causal effect is estimated using two-stage least squares. In the first stage, the 
dependent variable (completing the program) is regressed on the exogenous covariates plus the 
instrument (randomization into treatment). In the second stage, fitted values from the first-stage 
regression are plugged directly into the structural equation in place of the endogenous regressor 
(completing the program). In both stages, we used a linear random effects model. We included the same 
covariates from the ITT regression. 
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study Participants 


Overview of Baseline Characteristics 

This section presents the characteristics of the study sample overall and separately for the treatment 
and control groups (table 1). The study sample included 1,062 youths who applied to the Urban Alliance 
program and agreed to participate in the evaluation. 

The vast majority of study youth were African American (89 percent), with small shares of "other" 
race/ethnicity (4 percent), Hispanic (5 percent), and white (2 percent).^"^ Most of the study sample were 
US citizens (95 percent). Eleven percent reported being an English language learner. 

About two-thirds of the study sample was female. Given that the recruitment process was not 
aimed at either female or male students in particular, the factors underlying this discrepancy are 
unclear. However, we found other programs geared toward high school students also typically serve 
more female than male youth. For example, youth in After School Matters were 59 percent female 
(Hirsch et al. 2011), the Summer Career Exploration Program was 62 percent female (McClanahan, 
Sipe, and Smith 2004), and Upward Bound was 7 1 percent female (Myers et al. 2004).^^ 

Urban Alliance applicants typically come from underemployed households, with nearly a quarter of 
students reporting that no adults in their household were employed. Still, about three-quarters of 
applicants reported at least some prior work experience of their own, with average experience of just 
less than 10 months in all jobs combined. Most typically these positions were summer jobs, including 
jobs accessed through the Summer Youth Employment Program in Washington, DC. About 4 in 10 
youths reported having a checking or savings account, and a greater portion of youth with job 
experience (42 percent) than of youth with no job experience (30 percent) reported having an account. 
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TABLE 1 


Characteristics of Urban Alliance Applicants, Overall and by Treatment Group 


Characteristic 

Full sample 

Treatment 

Control 

Demographics 




Female 

65% 

66% 

63% 

Race 




African American 

89% 

88% 

91% 

White 

2% 

2% 

2% 

Hispanic 

5% 

6% 

4% 

Other 

4% 

4% 

3% 

US citizen 

95% 

95% 

97% 

English language learner 

11% 

11% 

11% 

Family characteristics 




Mother present in household 

82% 

83% 

80% 

Father present in household 

32% 

33% 

30% 

Has a child 

4% 

5% 

4% 

Employed adult in household 

77% 

77% 

78% 

Living Arrangement 




Father only 

5% 

5% 

5% 

Mother only 

56% 

56% 

57% 

Other 

12% 

11% 

13% 

Two parents 

27% 

28% 

25% 

Other characteristics 




Had a previous job 

75% 

76% 

74% 

Has a checking or savings account 

38% 

35% 

42% 

Money saved 

$99 

$94 

$108 

Observations (n) 

1,062 

700 

362 


Source: Urban Alliance high school internship program application forms. 

Notes All items had a response rate of 80 percent or more except bank account (71 percent). 
* significant at 10%; ** significant at 5%; *** significant at 1% 
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TABLE 2 


Characteristics of Urban Alliance Applicants’ Neighborhoods 



Baltimore 

Washington, DC 

Both sites 

Share nonwhite 

<25% 

4 

2 

2 

25% to <50% 

7 

4 

5 

50% to <75% 

12 

9 

9 

>75% 

79 

85 

84 

Share in poverty 

<10% 

9 

12 

11 

10% to <25% 

42 

39 

40 

25% to <40% 

40 

33 

34 

>40% 

11 

16 

15 

Share unemployed 

<5% 

3 

3 

3 

5% to <10% 

13 

21 

20 

10% to <20% 

55 

38 

41 

>20% 

30 

37 

36 

Observations (n) 

200 

846 

1,046 


Sources: Neighborhood characteristics are five-year averages at the tract level from the American Community Survey, 2008- 
2012, US Census Bureau. Tract determinations are based on youth addresses as reported on Urban Alliance high school 
internship program application forms. 

Notes: Estimates include applicants assigned to the treatment group and the control group. The table does not include 16 
applicants with incomplete address information. 


Applicants typically resided in economically distressed neighborhoods (table 2). More than three- 
quarters (77 percent) lived in a neighborhood with an unemployment rate greater than 10 percent, and 
nearly half lived in neighborhoods with poverty rates higher than 25 percent. Maps of program 
applicants’ home locations in Baltimore and Washington, DC, reveal that almost all reside in census 
tracts that are over 75 percent minority (dark green shading in figure 2). Most applicants resided in 
communities with higher-than-average poverty, though a sizable portion of applicants lived in 
moderate-poverty areas (medium-light blue shading). Almost no youths hailed from low-poverty 
sections of Baltimore or DC. As one executive staff member described, "some of them are living in 
moderate-income, mostly African American communities, [and] some of them are living in the toughest 
communities in the District.” Generally, staff members believe that youths’ upbringing in largely 
segregated and low-income neighborhoods limit their opportunities for socioeconomic mobility. As one 
senior member explained, most participants "haven’t left their neighborhood,” in the sense that they 
have had very little exposure to opportunities found in middle- and upper-class communities. 
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FIGURE 2 

Characteristics of Urban Alliance Applicants’ Neighborhoods 


Percentage poor in Baltimore 


Percentage nonwhite in Baltimore 




Percentage poor in Washington, DC 


Percentage nonwhite in Washington, DC 




Sources: Percentage nonwhite and percentage poor are five-year averages at the tract level from the American Community 
Survey, 2008-2012. Tract determinations are based on youth addresses as reported on Urban Alliance high school internship 
program application forms. 

Notes: Dots represent the number of program applicants within each census tract and are placed randomly within each tract to 
display the relative distribution of applicants across region. One dot = one applicant. 


Slightly more than a quarter of Urban Alliance applicants attended a charter school, most of whom 
were in Washington, DC, where about one-third (34 percent) of applicants attended charter schools. 
Applicants on average exhibited passing but not stellar performance in school, as shown in table 3. The 
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average cumulative GPAat the end of junior year was 2.7, according to school records if available, or as 
reported on the application by a school counselor or by the student. A small but nontrivial share of 
students (7 percent) participated in a special education program. Over one-third of applicants (37 
percent) had attended more than one high school, a pattern often characteristic of high household 
instability (Theodos, Coulton, and Budde 2014). Applicants demonstrated the intention of attaining a 
postsecondary degree, with over 90 percent indicating plans to take the SAT or ACT. 

TABLE 3 

Academic Achievement, Educational Attributes, and School Characteristics of Urban Alliance 
Applicants, Overall and by Treatment Group 


Characteristic 

Full sample 

Treatment 

Control 

Academic achievement and educational attributes 

Number of other schools attended in past three 
years 

0.5 

0.5 

0.5 

In special education 

9% 

8% 

10% 

GPA at end of junior year 

2.7 

2.7 

2.6 

Has taken or plans to take ACT or SAT 

90% 

90% 

91% 

Attends magnet school 

9% 

10% 

7% 

Attends charter school 

28% 

28% 

28% 

School-level characteristics 




Proficient or advanced in reading 

44% 

45% 

43% 

Proficient or advanced in math 

40% 

41% 

39% 

African American 

90% 

89% 

90% 

Hispanic 

6% 

7% 

6% 

White 

3% 

3% 

3% 

Other 

1% 

1% 

1% 

Eligible for free or reduced-price lunch 

77% 

77% 

79% 

Observations (n) 

1,062 

700 

362 


Sources: Urban Alliance high school internship program application forms for school; National Centerfor Education 
Statistics data from 2010 for student body demographic information; Maryland State Department of Education and the 
Washington, DC, Office of the State Superintendent of Education from 2011 for math and reading proficiency. GPAs, 
special education status, and some information on school transfers were provided by DC Public Schools, Baltimore 
Public School Board, DC Public Charter School Board, and individual charter schools in DC. 

Notes: All items had a response rate of 80 percent or more except "has taken or plans to take ACT or SAT" (72 percent). 
Variance is due to nonresponse for some items on the application form. 

* significant at 10%; ** significant at 5%; *** significant at 1% 


Urban Alliance applicants attended a mix of 38 public and charter schools. In DC, about two-thirds 
of youth attended one of eight schools, each accounting for 30 to 61 program applicants; in Baltimore, 
three-quarters attended one of seven schools, each serving at least 9 applicants. Most of the schools 
attended in both sites were low-performing and attended mostly by youth of color (see table 3). In fact. 


28 


EMBARKING ON COLLEGE AND CAREER 



almost all schools that Urban Alliance applicants attended were majority African American, though 
some schools in DC also had significant Hispanic student contingents as well. About 93 percent of youth 
attended schools with the majority of students eligible for free or reduced-price school lunch. 

Despite these commonalities, the schools attended differed in some ways. Two schools in DC 
enrolling large numbers of Urban Alliance applicants, Dunbar High School (39 youths) and McKinley 
Technology High School (61 youths), exemplify the diversity in school characteristics. Both have over 
95 percent African American student bodies, but at Dunbar, 100 percent of students were eligible for 
free or reduced-price lunch and barely a quarter of students were proficient on District-wide reading 
and math exams. At McKinley Tech, however, just over half the students were eligible for free or 
reduced-price lunch and nearly 90 percent were proficient in math and reading. Overall, 45 percent of 
Urban Alliance applicants attended a school ranking in the bottom quartile of proficiency in reading and 
math in DC or Maryland, with fewer than 10 percent of students at a school in the top quartile. 

Differences between Treatment and Control Groups 

Data from the Urban Alliance application and school data indicate there were few differences across 
the treatment and control groups, as shown in tables 1 and 3. The overall comparability of these two 
groups indicates that randomization was successful for this study. The variables that were different 
statistically at baseline include the following: US citizenship (95 percent treatment, 97 percent control); 
having a bank account (42 percent treatment, 35 percent control); average GPA at the end of junior 
year (2.7 for treatment, 2.6 for control);^^ attending a magnet school (10 percent treatment, 7 percent 
control); the schools’ percentage proficient or advanced in math (41 percent treatment, 39 percent 
control); and the schools’ percentage eligible for free or reduced-price lunch (77 percent treatment, 79 
percent control). As can be seen, even when there is statistical significance, the differences are 
generally not large. 

Baseline characteristics of youth that completed the follow-up survey, overall and by treatment and 
control group, are shown in appendix B. 

Differences across Sites 

Across sites, there were a moderate number of differences in applicants, illustrated in table 4. The vast 
majority of students at each site were African American, but there was a higher percentage of white 
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applicants in Baltimore and a higher percentage of Hispanic applicants in DC. There were many more 
English language learners in DC (13 percent) than in Baltimore (1 percent). 

TABLE 4 

Characteristics of Urban Alliance Applicants, by Site 


Characteristic 

Baltimore 

Washington, DC 

Youth demographics 



Female 

63% 

65% 

Race 



African American 

91% 

89% 

White 

4% 

1% 

Hispanic 

2% 

6% 

Other 

3% 

4% 

US citizen 

97% 

95% 

English language learner 

1% 

13% 

Youth family characteristics 



Mother present in household 

81% 

82% 

Father present in household 

31% 

32% 

Idas a child 

5% 

4% 

Employed adult in household 

78% 

77% 

Living arrangement 



Father only 

6% 

5% 

Mother only 

56% 

56% 

Other 

13% 

12% 

Two parents 

25% 

27% 

Other characteristics 



Had a previous job 

75% 

75% 

Has a checking or savings account 

38% 

38% 

Money saved 

$243 

$65 

Academic achievement and educational attributes 



Number of other schools attended in past three years 

0.6 

0.4 

In special education 

12% 

8% 

GPA at end of junior year 

2.7 

2.6 

Has taken or plans to take ACT or SAT 

89% 

91% 

Observations (n) 

201 

861 


Source: Urban Alliance high school internship program application forms. 

Notes: All items had a response rate of 80 percent or more except bank account (71 percent) and “has taken or plans to 
take ACT or SAT" (72 percent). Variance is due to nonresponse for some items on the application form. 

* significant at 10%; ** significant at 5%; significant at 1%; +++ chi-squared test significant for categorical variable at 
the 1% level 
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Family structure was very similar across sites. Youth in DC were no more or less likely to have 
either a father or mother present in the household than youth in Baltimore, and living arrangements 
were similar. 

Youth in DC reported having saved significantly less money ($65) than their Baltimore 
counterparts ($243). Additionally, youth in Baltimore reported having attended a greater number of 
high schools (other than their current school) in the past three years. 
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Program Take-Up and Services 
Received 


To understand better the impacts of the Urban Alliance, we first explored the level of participation in 
the program and students’ reasons for not completing the program. The Urban Alliance model allows 
youth to self-select into the program and, by design, expects significant attrition. The Urban Alliance 
program has three stages at which a student could self-select out of the program. First, the student 
could not show up to pre-work training. Second, the student could elect not to complete pre-work 
training, which lasts for three to six weeks. Finally, the student could fail to complete his or her 
internship. 

Program Attrition 

Attrition was substantial among participants in the Urban Alliance internship program. Attrition 
primarily occurred in the first two stages, before and during pre-work training. Of those assigned to the 
treatment group, 22 percent did not attend any pre-work sessions. A quarter (25 percent) began but did 
not complete pre-work training. Urban Alliance placed the remaining 52 percent in a job, and most (84 
percent) of those students completed the program. I n total, 41 percent of treatment group youth 
completed the program.^® 

In both sites and years, the majority of the attrition occurred prior to completing pre-work training 
(table 5). Although DC and Baltimore had similar rates of program completion (42 and 40 percent, 
respectively), the attrition occurred at different stages in the program. In DC, most attrition occurred 
after beginning pre-work but before completing pre-work. In Baltimore, most attrition occurred before 
attending any pre-work sessions. There were also differences across cohorts, with most attrition 
occurring after the first pre-work session in 2011-12 and most attrition occurring before the first pre- 
work session in 2012-13. The overall rates of completion were higher in 2011 than in 2012. These 
differences across sites and years may have been the result of differences in recruitment and pre-work 
training methods (Theodos et al. 2014). 

Of youth who completed an internship, about a quarter (26 percent) engaged with Urban Alliance 
as alumni by working with alumni services staff to resolve a school, job, or personal issue; participating 
in Urban Alliance’s summer College Internship Program (in DC); or being linked to a summer internship 
informally (in Baltimore). Sixteen percent of youth completing an internship participated in a summer 
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internship, 13 percent engaged alumni services, and 4 percent did both. Internship participation was 
nearly identical for both cohorts, but more program completers worked with alumni services staff in 
2011 than in 2012. More program completers in Baltimore (17 percent) worked with alumni services 
staff than in DC (13 percent), but participation in internships was much higher in DC (19 percent) than 
Baltimore (4 percent), because DC has a formal program and Baltimore does not. 

TABLE 5 

Attrition, by Site and Cohort 



All 

2011-12 
(both sites) 

2012-13 
(both sites) 

DC 

(both cohorts) 

Baltimore 
(both cohorts) 

Application accepted (/?) 

700 

310 

390 

581 

119 

Attended pre-work 

78% 

88% 

69% 

79% 

70% 

Completed pre-work 

52% 

62% 

45% 

51% 

57% 

Placed at a job 

49% 

59% 

40% 

47% 

55% 

Completed program 

41% 

51% 

33% 

41% 

42% 


Source: Urban Alliance program data. 

Note: “Completed program’’ is defined as remaining in the Urban Alliance program until June 1. 


Predicting Take-Up 

As described above, treatment group youth could drop out of the Urban Alliance program atone of 
three main stages: they could elect not to show up to pre-work training, not complete the pre-work 
sessions, or not complete the internship.^^ It is possible to generate predictive models that relate 
baseline characteristics of the youth and program to the likelihood that youth will complete each of 
these stages. The baseline characteristics we explored include gender, family structure, speaking a 
language other than English at home, neighborhood poverty levels, employment experience, taking 
special education courses, parenting, GPA, cohort (2011-12 or 2012-13), site (DC or Baltimore), and 
program coordinator caseload. 

This section describes results from four models. The first examines which baseline factors are 
associated with the probability of attending pre-work. The second explores the baseline characteristics 
that predict completing pre-work, conditional on attending pre-work. That is, the second model 
examines only those youth who showed up to pre-work, and then looks at the factors associated with 
their completing pre-work. The third model examines the characteristics predicting program 
completion, conditional on completing pre-work. Finally, we report on a fourth model: the probability of 
completing the program overall, unconditional on having completed pre-work. This final model does not 
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account for which stage a youth dropped out of the program, but rather, informs more globally how the 
41 percent of treatment group youth who completed differed from the 59 percent of treatment group 
youth who did not. 

In the first model— probability of attending pre-work— five significant predictors of attending pre- 
work were found (table 6): neighborhood poverty, previously holding a job, taking special education 
courses, parenting, and cohort 2011-12. Having previously held a job was positively associated with 
attending pre-work. Those who previously held a job had a probability of attending pre-work roughly 9 
percentage points higher than their counterparts. Taking special education courses and being a parent 
were both negatively associated with attending pre-work. These indicators may represent barriers to 
attending pre-work based on lack of child care or having an inflexible school schedule. As described 
above and verified here, the program year 2011-12 cohort was associated with a higher rate of 
showing up to pre-work than was the cohort enrolled during program year 2012-13. 

Next, in the model predicting completion of pre-work among those who showed up to pre-work, we 
see only a few significant baseline predictors. Although they were no less likely to show up to pre-work, 
youth in DC were less likely to complete pre-work relative to Baltimore youth. GPA mattered as well. 
The probability of completing pre-work training was 26 to 28 percentage points higher for youth with 
GPAs between 2.0 and 4.0 than for those with GPAs below 2.0. Although youth with a GPA of 2.0 or 
lower were accepted by Urban Alliance, they may have had trouble remaining in the program because 
they needed to focus more on their studies to ensure they graduated from high school. Alternatively, 
GPA could be a proxy for a more general measure of motivation. 

In the third model— probability of completing the internship conditional on having completed pre- 
work— again relatively few baseline factors were successful in predicting internship completion. 
Interestingly, although DC youth were less likely to complete pre-work than those in Baltimore, they 
were more likely to successfully complete their internship. The only other significant predictor of 
completing the program, among those youth who completed pre-work training, was the caseload of the 
program coordinator. A reduction in the caseload of a youth's program coordinator by 10 youths 
decreased the probability of attrition at the internship stage by 9 percentage points. 

The caseload result is understood in light of the fact that the program coordinators’ role was to help 
students through the program. Most students met with their program coordinator at least once during 
the program and received help from them in applying to college. The program coordinators managed a 
caseload of anywhere from 4 to 44 students. In interviews as part of the process study, senior staff 
expressed that a caseload of 30 to 35 was considered the program's target. Program coordinators 
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themselves reported they would prefer their caseloads to be smaller— one offered 20 to 25 students as 
ideal. Program coordinators described their positions as a lot of work, fast-paced, and often 
overwhelming— that the job felt like "two years in one,” was "beyond nine-to-five,” and was "more than a 
teacher's workload.” Despite these complaints, most program coordinators also described the job as 
allowing for a good work-life balance. 

TABLE 6 

Probability of Take-Up 


Probability of 

Probability of 


attending pre- 

completing 

Probability of completing the 

work 

pre-work 

program 


Conditional on 

Conditional on 


attending pre- 

completing 


Variable 

Unconditional 

work 

pre-work 

Unconditional 

Female 

0.0474 

-0.0585 

0.0106 

0.00199 


(0.0325) 

(0.0455) 

(0.0463) 

(0.0403) 

Student is a parent 

-0.157” 

-0.147 

0.0625 

-0.122 


(0.0691) 

(0.112) 

(0.135) 

(0.0989) 

Previously held job 

0.0860” 

0.0226 

-0.0388 

0.0392 


(0.0376) 

(0.0522) 

(0.0576) 

(0.0469) 

Single-parent family 

-0.0248 

-0.00661 

0.00318 

0.00540 


(0.0399) 

(0.0505) 

(0.0532) 

(0.0455) 

Other family structure 

-0.0502 

0.0636 

-0.101 

0.00227 


(0.0579) 

(0.0801) 

(0.0711) 

(0.0704) 

Parents speak language other than 
English 

-0.0302 

-0.0145 

0.0446 

0.0318 


(0.0581) 

(0.0855) 

(0.0956) 

(0.0745) 

Percentage poverty in neighborhood 

0.00243* 

0.000642 

-0.000529 

0.00106 


(0.00135) 

(0.00170) 

(0.00172) 

(0.00157) 

Taking special education courses 

-0.142*** 

-0.0106 

0.00713 

-0.0980 


(0.0547) 

(0.0884) 

(0.0956) 

(0.0760) 

GPA: 3 to 4 

-0.0306 

0.283*** 

0.0653 

0.189*** 


(0.0510) 

(0.0692) 

(0.0823) 

(0.0698) 

GPA: 2 to <3 

0.0430 

0.257*** 

0.0710 

0.230*** 


(0.0474) 

(0.0635) 

(0.0777) 

(0.0648) 

2011-12 cohort 

0.186*** 

0.0460 

0.108** 

0.139*** 


(0.0357) 

(0.0459) 

(0.0549) 

(0.0402) 

DC 

0.0425 

-0.220*** 

0.173* 

-0.0576 


(0.0538) 

(0.0797) 

(0.0902) 

(0.0710) 

Caseload 



-0.0881** 


(0.0442) 

Observations [n] 

690 

520 

360 

690 


Sources: Urban Alliance high school internship program application forms and Urban Alliance program data. 

Notes: Estimates are marginal effects from a logit regression. Standard errors are given in parentheses. "Completing the program" 
is defined as remaining in the Urban Alliance program until June 1 . Reference group for GPA 3.0 to 4.0 and GPA 2.0 to <3.0 is 
GPA <2.0. 

*p < 0.1 ”p< 0.05 ”*p<0.01 
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When predicting the probability of program completion, unconditional on completing the previous 
steps, the only significant predictors were GPA and cohort. To be clear, it appears that youth in DC were 
no more or less likely to complete the program than youth in Baltimore, nor did completion vary by 
gender, parenting, employment history, family structure, language spoken at home, neighborhood 
poverty, or taking special education courses. (We cannot test for caseload in this regression, because 
program coordinators were not assigned to youth before pre-work.) As in the second model, higher 
CPAs were positively associated with completing the internship. The probability of completing the 
program was 19 to 23 percentage points higher for youth with CPAs between 2.0 and 4.0 than for those 
with CPAs below 2.0. Additionally, youth from the 2011-12 cohort were 14 percentage points more 
likely to complete the program than youth from the 2012-13 cohort. 

Reasons for Attrition 

By design. Urban Alliance expects some attrition from its program. Most youth apply to the program 
months before the start of pre-work training, so their interest or ability to commit to the program may 
change by the time the program starts. Moreover, Urban Alliance expects that some youth, when they 
begin pre-work training, will realize they do not wish to continue in the program; for this reason it 
accepts more youth into the program than it has internship slots for. The majority of attrition that 
occurs— before and during pre-work training— is anticipated. 

In our discussions with students, we discovered several reasons that a student would not complete 
the program. Scheduling was among the most commonly cited reasons for attrition, especially in the 
pre-work training phase. The program requires that youth participate in work or a workshop from 2:00 
to 5:00 p.m. each day during the school year. Some students cited competing priorities such as athletics 
and other after-school activities. These extracurricular scheduling conflicts often became clear very 
early into program participation, contributing significantly to attrition in the pre-work training phase. 
For some students, school class schedules were an impediment to participation, which also led to 
attrition primarily in the pre-work training phase. Although in recent years Urban Alliance has 
attempted to verify school schedules earlier in the recruitment and application process to be certain 
youth are eligible, conflicts still arise. Even once the school year begins, many schools were unsure of 
youth eligibility for an early-release schedule. In some instances, Urban Alliance invited youth to pre- 
work training who were ultimately unable to have their schedules rearranged to participate in an 
internship. Other youth could have altered their schedules but opted not to. Other causes of attrition 
during pre-work training reported by youth included a lack of motivation or interest to complete 
training, personal or family issues preventing continued participation, and relocation. 
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Youth also cited the cost of transportation to and from training events and job sites as a reason for 
attrition. Transportation primarily contributes to attrition before the internship phase of the program, 
but it remains a significant financial and scheduling difficulty for many students throughout the length 
of their internships. 

Although most attrition occurs during pre-work training and before the internship phase, there was 
a drop-off during internships as well. Of the 343 youths in the study who began an internship, 57 youths 
did not complete the program. Among those for whom the reason for departure is known, 20 quit and 
31 were fired. Of those who were fired, the primary reason was usually attendance (15 youths) or 
misconduct (10 youths), which frequently involved time-sheet fraud. Five were fired for poor 
performance, and one was asked to leave because the program was interfering with her academics. 
Among those who quit, common reasons were wanting to pursue another job or educational 
opportunity (6 youths), having a personal or family obligation that took precedence over the program (6 
youths), needing more time for high school studies (4 youths), and disliking the internship (4 youths). 
Though Urban Alliance considered job site location and distance from school when placing a student, 
some youth who did not complete the program cited long transit times as their reason for quitting. 

Services Received 

If the control group in an experimental evaluation receives similar services as the treatment group, then 
it is unlikely that researchers will observe a difference in outcomes of the individuals in these two 
groups. To test whether treatment youth were in fact more likely to report receiving employment and 
education support services than the control group, we asked in the outcome survey about whether 
youth had received any job or college help through a class or workshop. We asked both treatment and 
control youth about the college and job preparation help they received, which is an important 
consideration because the Urban Alliance program is not operating in isolation. Rather, many high 
schools and other after-school programs now provide some form of career and college support and 
planning. Understanding non-Urban Alliance support and programming before examining 
programmatic impacts is important because in an RCT impacts are assessed for the treatment group 
relative to the control group, and to the extent the control group received comparable services, 
programmatic impacts will be diminished. One additional note is that although we measured the receipt 
of such services, we had no ready way to measure the quality of the services provided, and it is certainly 
plausible that not all of these services were equally impactful. 
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Table 7 displays the results of this analysis. The "Mean” columns show the average share of youth in 
the treatment group and the average share of youth in the control group receiving job help or college 
help. We found that the majority of youth in the treatment and control groups received job help and 
college help. The data under the "ITT" heading show the uncontrolled difference in means and the 
regression-adjusted difference in means. 

The treatment group was more likely than the control group to report receiving college help and job 
help in both the uncontrolled difference and the regression-adjusted difference. However, the 
differences for college help and job help were relatively small under the ITT framework, only 8 and 12 
percentage points, respectively. This small difference reflects the high rates of control group youth 
receiving college help (85 percent) and job help (82 percent). 

We also explored how receipt of job and college help varied for different subgroups of study 
participants: males and females; DC and Baltimore; and those with low, middle, and high GPAs. The 
results are shown in tables in appendix C. The ITT difference in reported college help in Baltimore was 
much higher (18 percentage points) than in DC (6 percentage points). That is, in DC the treatment group 
reported getting help with college at a rate only 6 percentage points higher than the control group, but 
in Baltimore the treatment group reported getting help at rates that were 18 percentage points higher 
than in DC. This difference partly reflects the decreased likelihood of youth in the control group 
reporting receiving any college help in Baltimore, with only 79 percent reporting receiving college help 
compared to 87 percent of the control group in DC. There was no substantial difference between DC 
and Baltimore in reporting receiving job help for either the treatment or control groups. 

Males in the control group reported receiving moderately less college and job help at 81 and 78 
percent, respectively, than females in the control group at 88 and 84 percent, respectively. There was, 
however, no difference in the impacts of the program on reporting receiving college help between males 
and females. Finally, the rate of receiving college help was especially lowfor those with GPAs below 2.0, 
and it was this group that had a higher likelihood of receiving job help. 
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TABLE 7 


Services Received 


Mean ITT 

Difference Regression 

Outcome Treatment Control in means adjusted 

Received job help 0.95 0.82 0.12"* * 0.128*** 

(0.031) 

Received college help 0.93 0.85 0.08*** 0.081*** 

(0.027) 


Sources: Urban Alliance program data (control variables) and interim outcome survey (outcome variables). 

Notes: Robust standard errors are given in parentheses. Intent to treat (ITT) compares outcomes of a treatment group of 
individuals who were accepted Into the program (but who may or may not have completed the internship) with a control group of 
individuals who were not accepted into the program. The regression-adjusted models included the following control measures: 
program year, gender, neighborhood percentage poverty, previously held a job, and junior year GPA. 

* significant at 10%; ** significant at 5%; *** significant at 1% 
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Program Impacts 

In this section, we present impact findings of Urban Alliance for educational and employment outcomes. 
Under each outcome we present the results under both the regression-adjusted ITT and TOT 
framework. We controlled for demographic and educational characteristics in each of the regressions. 
Although youth outcomes within a school may vary, youth from the same schools often have similar 
educational outcomes. To account for within-school correlation, we estimated a random-effects model 
at the school level. In each table, we present the results showing the mean for the treatment and control 
group, the uncontrolled difference in means, and the coefficients from each regression, identifying the 
level of statistical significance achieved for each. We present the findings on the effect of Urban 
Alliance on education preparation and job preparation, education advancement, employment and 
wages, and savings. 

In addition to estimating the impacts on the full population of Urban Alliance participants in 
program years 2011-12, we estimated the impacts separately for each site and conducted two 
subgroup analyses, by site, by junior-year cumulative GPA, and by gender. Because key differences 
existed between DC and Baltimore, including how the Urban Alliance program was implemented, the 
youth enrolled, and the environment, we ran the regressions separately for the two sites. One of the key 
targeting criteria identified by the program is "middle of the road” students which we defined as 
students with CPAs above 2.0 but below 3.0. To test whether the program was more effective for youth 
in their target population, we conducted a subgroup analysis on three groups of youth: those with a 
cumulative junior year GPA below 2.0, between 2.0 and 3.0, and above 3.0. The gender gap in GPA and 
college attendance has been widely documented (Jacob 2002; Voyer and Voyer 2014). We found a 
similar gender gap in our control group's GPA, high school graduation rate, and college attendance. In 
addition, we found that males in the control group were less likely to receive college or job help than 
females in the control group. These differences may translate into differential effects of the program by 
gender. For these reasons, we conducted a subgroup analysis for males and females. The results of each 
of these subgroup analyses can be found in appendix C. 

Education and Employment Preparation 

Training and mentoring, which aim to prepare students for college and employment, are key program 
components. To assess preparation for college, we estimated the impact of the program on six 
education preparation measures: taking the SAT, taking the ACT, filling out the FAFSA, comfort 


40 


EMBARKING ON COLLEGE AND CAREER 



completing the FAFSA or scholarship applications, identifying grants and scholarships to help pay for 
college, and applying to college. All of these self-reported measures are drawn from the outcome 
survey. 

We did not detect programmatic impacts on the probability of taking the SAT or filling out the 
FAFSA (table 8). However, finding gains in these areas would prove challenging, as more than 91 
percent of control group youth reported doing both. It appears that taking the SAT and completing the 
FAFSA for youth in Baltimore and DC are near universal experiences— at least for those applying to the 
Urban Alliance program— though Urban Alliance staff believe it is common foryouth to report 
completing the FAFSA without actually having done so. Treatment group youth, however, were more 
likely to take the ACT. Those applicants offered treatment took the ACT at a rate that was roughly 7 
percentage points higher than control group youth; this difference was significant at the .05 level. 
However, after adjusting for multiple comparisons, the difference was only significant at the .10 level. 

We also found the program improved youth comfort with filling out the FAFSA and scholarship 
applications. The average comfort of those youth offered treatment was 0.13 points higher (on a four- 
point scale) than the comfort of youth in the control group, and those who completed the internship had 
an average comfort 0.26 points higher than those in the control group. After adjusting for multiple 
comparisons, the difference between the treatment and control groups was no longer significant. More 
than 90 percent of the control group took the SAT and filled out the FAFSA, which may explain why we 
did not detect treatment-control differences. We did not find programmatic impacts on the probability 
of applying to college, though as with the SAT and FAFSA completion, the control group reported 
applying for col lege at high rates (92 percent). 

We used three impact measures to assess job preparation. The first is an average of a scale of youth 
comfort with writing a cover letter or resume, completing a job application, asking someone to serve as 
a job reference, and being interviewed for a job. We labeled this measure "job application comfort." 
Although this scale has not been validated, it hangs together well, with a Cronbach’s alpha of 0.77. We 
also estimated the impact of the program on youths' hard and soft skills. We measured hard skill 
comfort as their self-reported comfort with performing general office work. Soft skill comfort was an 
average of a scale of self-reported comfort with speaking with and writing e-mails to professionals, 
making a presentation, dressing professionally, completing work assignments on time, and getting to 
work on time. This scale also has not been validated, but has acceptable internal consistency with a 
Cronbach’s alpha of 0.76. Indeed, such skills are quite important. Agrowing literature has demonstrated 
the value of both hard and soft skills for a youth’s academic and employment success (Lippman et al. 
2015). 


PROGRAM IMPACTS 


41 



We did not detect an impact on overall job application comfort, although we found that Urban 
Alliance increased both youths' hard skill comfort and soft skill comfort. Based on the ITT regression- 
adjusted results, the program increased hard skill comfort by 0.14 and soft skill comfort by 0.10 on a 
four-point scale for youth offered access to the program relative to the control group. With the 
adjustment for multiple comparisons, the effects remained, but they were significant at the .10 level 
rather than the .05 level. For treated youth, there were gains of 0.25 in hard skill comfort and 0.20 in 
soft skill comfort. 

TABLES 

Education and Employment Preparation Impacts 


Mean ITT TOT 


Outcome 

Treatment 

Control 

Difference 
in means 

Regression 

adjusted 

IV regression 
adjusted 

Education preparation 

Took SAT 

0.91 

0.91 

-0.004 

-0.015 

-0.038 





(0.022) 

(0.049) 

Took ACT 

0.44 

0.35 

0.091" 

0.071* 

0.132 





(0.041) 

(0.084) 

Filled out FAFSA 

0.93 

0.92 

0.010 

-0.004 

-0.008 





(0.021) 

(0.046) 

Comfort with FAFSA and 
scholarships 

3.59 

3.47 

0.129" 

0.129" 

0.262** 





(0.052) 

(0.109) 

Applied to college 

0.94 

0.92 

0.023 

0.010 

0.011 





(0.021) 

(0.043) 

Job preparation 

Job application comfort 

3.64 

3.58 

0.060 

0.060 

0.125 





(0.043) 

(0.088) 

Hard skill comfort 

3.66 

3.52 

0.142" 

0.124** 

0.254** 





(0.058) 

(0.116) 

Soft skill comfort 

3.74 

3.63 

0.106"* 

0.099*** 

0.203*** 





(0.035) 

(0.073) 


Sources: Urban Alliance program data (control variables) and interim outcome survey (outcome variables). 

Notes: IV = instrumental variables. Robust standard errors are given in parentheses. Intent to treat (ITT) compares outcomes of a 
treatment group of individuals who were accepted into the program (but who may or may not have completed the internship) 
with a control group of individuals who were not accepted into the program. T reatment on the treated (TOT) compares outcomes 
of those in the treatment group who completed the internship to those in the control group. The regression-adjusted models 
included the following control measures: program year, gender, neighborhood percentage poverty, previously held a job, and 
junior year GPA. 

* significant at 10%; ** significant at 5%; significant at 1% 
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DC had similar findings to the overall results, with a few exceptions. The DC site saw positive 
impacts on comfort completing the FAFSA and applying for scholarships (though this impact 
disappeared after adjusting for multiple comparisons), hard skill comfort, soft skill comfort, and job 
application comfort. There were no detectable impacts on taking the ACT, however. Apart from comfort 
with hard skills, there were no significant differences in Baltimore in the college and education 
preparation measures, potentially because of the small sample size. The effect on hard skills remained 
after adjusting for multiple comparisons, though at a lower level of significance (0.10). 

Results for females mirrored the group as a whole, though impacts on taking the ACT and on 
comfort completing the FAFSA and applying for scholarships were not present after adjusting for 
multiple comparisons. All the estimates for males were positive, but none were statistically significant. 

We did not detect any positive impacts on education and career preparation for the low-GPA 
(below 2.0) group, potentially because of the small sample size. For the middle-GPA (2.0 to less than 3.0) 
group, the treatment group reported significantly higher soft skill comfort— at the .10 level after 
adjusting for multiple comparisons— though not hard skill comfort. The treatment group in the 3.0 to 4.0 
GPA subgroup reported significantly higher soft skill comfort as well, and also reported higher comfort 
completing the FAFSA and applying for scholarships. 

Education Achievement 


High School 

To test whether Urban Alliance affected educational outcomes while students were still in high school, 
we evaluated the effect of Urban Alliance on the probability of high school graduation, suspension 
senior year, chronic absence senior year, and cumulative GPA through senior year.^° We found no 
significant impacts under the ITT or TOT framework (table 9). Because youth begin interacting with 
Urban Alliance in the fall of their senior year, many of these measures, particularly cumulative GPA and 
probability of high school graduation, were already largely determined prior to the program's start. 

We did not find any impacts on high school outcomes at either site. However, males in the 
treatment group were 4 percentage points more likely to graduate high school than control group 
males, though this difference was not statistically significant after adjusting for multiple comparisons. 
The males in the control group had a lower high school graduation rate (94 percent) compared to the 
females in the control group (97 percent). Ninety-nine percent of males in the treatment group 
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graduated high school, and 100 percent of those who completed the program graduated high school. 
Males report receiving less job or college help in high school than females; thus, Urban Alliance likely 
has a greater opportunity to make a difference for males than females. 

TABLE 9 

Education Impacts: High School Achievement 


Outcome 

Mean 

ITT 

TOT 

Treatment 

Control 

Difference 
in means 

Regression 

adjusted 

IV regression 
adjusted 

Graduated high school 

0.98 

0.96 

0.022* 

0.012 

0.023 





(0.010) 

(0.025) 

Suspended senior year 

0.08 

0.10 

-0.019 

-0.011 

-0.036 





(0.018) 

(0.045) 

Chronically absent senior year 

0.32 

0.31 

0.009 

0.048 

0.111 





(0.031) 

(0.070) 

Cumulative GPA 

2.66 

2.55 

0.101** 

-0.012 

-0.027 





(0.026) 

(0.060) 


Sources: Urban Alliance program data (control variables) and DC Public Schools, Baltimore Public School Board, DC Public 
Charter School Board, and individual charter schools in DC (outcome variables). 

Notes: IV = instrumental variables. Robust standard errors are given in parentheses. Intent to treat (ITT) compares outcomes of a 
treatment group of individuals who were accepted into the program (but who may or may not have completed the internship) 
with a control group of individuals who were not accepted into the program. Treatment on the treated (TOT) compares outcomes 
of those in the treatment group who completed the internship to those in the control group. The regression-adjusted models 
included the following control measures: program year, gender, neighborhood percentage poverty, previously held a job, and 
junior year GPA. 

* significant at 10%; ** significant at 5%; *** significant at 1% 


College Attendance and Quality 

Urban Alliance aims to prepare youth for postsecondary education and employment. We explored the 
extent to which Urban Alliance affected participants’ college attendance and the quality of the colleges 
they attended. Specifically, using NSC data, we measured the share of youth who attended any college, 
the share attending a four-year college, and the share attending a two-year college. With IPEDS data 
linked to NSC attendance data we assessed three college quality measures: the 75th percentile SAT 
score for colleges that youth enrolled in, as well as those colleges’ retention rates and graduation rates. 

For the sample overall, we did not detect impacts on any of the college attendance measures, 
including both two- and four-year schools, under either the ITT or TOT framework (table 10). Sixty-six 
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percent of treatment group youth and 74 percent of treated youth attended college versus just 60 
percent of control group youth. However, these differences were not statistically significant after 
introducing baseline controls (i.e., in the regression-adjusted models). Similarly, 55 percent of treatment 
group youth and 62 percent of treated youth went to a four-year college, compared with 46 percent of 
those in the control group; differences were not statistically significant after introducing baseline 
controls. 

TABLE 10 

Education Impacts 

College Enrollment 


Mean ITT TOT 


Outcome 

Treatment 

Control 

Difference 
in means 

Regression 

adjusted 

IV regression 
adjusted 

Attended college 

0.66 

0.60 

0.065” 

0.029 

0.062 





(0.033) 

(0.072) 

Attended four-year college 

0.55 

0.46 

0.089*” 

0.035 

0.081 





(0.033) 

(0.072) 

Attended two-year college 

0.14 

0.16 

-0.019 

-0.008 

-0.009 





(0.023) 

(0.055) 

75th percentile SAT score of 
college attended 

1,048.06 

1,011.83 

36.230** 

30.025* 

50.648 





(16.391) 

(34.504) 

Retention rate of college 
attended 

65.41 

63.24 

2.169* 

0.694 

1.515 





(1.106) 

(2.419) 

Graduation rate of college 
attended 

35.97 

32.54 

3.429** 

1.506 

3.229 





(1.558) 

(3.427) 


Sources: Urban Alliance program data (control variables) and NSC and IPEDS (outcome variables). 

Notes: IV = instrumental variables. Robust standard errors are given in parentheses. Intent to treat (ITT) compares outcomes of a 
treatment group of individuals who were accepted into the program (but who may or may not have completed the internship) 
with a control group of individuals who were not accepted into the program. Treatment on the treated (TOT) compares outcomes 
of those in the treatment group who completed the internship to those in the control group. The regression-adjusted models 
included the following control measures: program year, gender, neighborhood percentage poverty, previously held a job, and 
junior year GPA. 

* significant at 10%; ** significant at 5%; *** significant at 1% 


We did see, however, that Urban Alliance had a positive and significant impact on the quality of 
college attended, as measured by the college’s 75th percentile SAT score, for participants who attended 
colleges that reported this metric. Not all colleges report SAT scores in the IPEDS database; this 
measure was missing for 224 of the 552 youths attending four-year schools. As a result, this estimate 
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may be subject to bias, as only those who attended colleges reporting 75th percentile SAT scores were 
included in the regression. However, we believe the selection mechanism would tend to bias this result 
downward, that is, if Urban Alliance pushes youth who do not attend college or who attend a two-year 
college to instead attend a four-year college on the low end of the quality scale. To control for this 
possibility, we also estimated a Heckman model to control for selection and obtained similar results; 
that is, that being offered access to the Urban Alliance program is associated with attending colleges 
with higher SAT scores. When an adjustment was made for multiple comparisons, however, the 
regression-adjusted ITT impact was no longer significant. 

In the analysis by site, the DC results only partially mirrored the overall findings. We did not detect 
significant impacts on college attendance in DC in the regression-controlled models, even though some 
of these differences were significant in the simple difference in means test. Youth invited to participate 
in the Urban Alliance program attended colleges with 75th percentile SAT scores that were 34 points 
higher than the control group, though the effect was not significant when accounting for multiple 
comparisons. We did not detect any college attendance impacts in Baltimore. 

An interesting pattern emerged when we examined college attendance impacts separately for 
females and males. Though our sample contained twice as many females as males, we did not find any 
effects on college attendance or quality for females. However, the program demonstrated large impacts 
on males in the probability of attending college and the probability of attending a four-year college. 
Males in the treatment group were 11 percentage points more likely to attend college than were 
control group males, and those completing the program were 26 percentage points more likely to 
attend college than their control group counterparts. This finding was driven by males being more likely 
to attend four-year rather than two-year schools: males offered access to the program were 12 
percentage points more likely to attend a four-year college and those completing the program were 27 
percentage points more likely to attend a four-year college compared with males in the control group. 
The effects on college attendance and four-year college attendance for males in the treatment group 
lost a level of significance (from 0.05 to 0.10) after adjusting for multiple comparisons. 

Examining the subgroup means offers further insights into these trends. Among females, 66 percent 
in the control group attended college compared with 67 percent in the treatment group and 72 percent 
in the treated group (i.e., those completing the program). Among males, just 50 percent of those in the 
control group went to college. Sixty-four percent of those in the treatment group and 76 percent of 
those completing the program attended a four-year college. In short, it appears that the Urban Alliance 
program benefited males by raising their rate of college attendance to that which females reach without 
the program. 
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Urban Alliance had no significant impact on college attendance or quality for the low-GPA 
subgroup. College enrollment was low for that group— 37 percent for the control group, 37 percent for 
the treatment group, and 48 percent for the treated group. 

However, we saw notable gains for students in the middle-GPA (2.0 to below 3.0) subgroup. Indeed, 
this is the group that the Urban Alliance program was designed to serve, though the program does not 
restrict admission to those with lower or higher GPAs. For the middle GPA subgroup, those youth 
offered treatment and those completing the internship were both more likely to attend a four-year 
college than the comparison group (by 12 and 21 percentage points, respectively). The impact for those 
offered treatment lost a level of significance after adjustments for multiple comparisons, from 0.05 to 
0 . 10 . 

Interestingly, the likelihood of attending a two-year college was lower for the treatment and 
treated groups than the control group. In this way, it appears that the Urban Alliance program helped 
middle-performing students who otherwise might have attended two-year colleges to instead attend 
four-year colleges. This result may explain why those youth offered the program and those who 
completed an internship went to colleges with higher graduation rates, as recorded in the IPEDS data, 
than did those in the control group. 

The program did not demonstrate any college attendance or quality impacts for high-performing 
youth (i.e., those with GPAs 3.0 or greater). It appears that college attendance was already prevalent for 
this group: 77 percent of control group youth with GPAs 3.0 or higher went to college. Eighty-one 
percent of youth offered access to the internship program and 84 percent of those completing the 
program enrolled in college. Seventy-two percent of control group youth in this GPA range attended a 
four-year college compared with 69 percent of youth offered access to the internship program and 71 
percent of those completing the internship program. 

Employment, Wages, and Savings 

In addition to preparing youth for college. Urban Alliance also prepares youth for employment. 
However, it is likely that positive labor market outcomes will develop over a longer time horizon than 
will positive education outcomes. This situation is especially the case if most youth enroll in college, 
because, as a result, many will not enter full-time work. 

Indeed, we did not find at this juncture that Urban Alliance had significant impacts on job 
attainment or wages after high school (table 11). This finding is likely the result of the high level of 
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postsecondary education enrollment among the treatment and control groups (77 and 71 percent, 
respectively). Significant differences between the treatment and control groups in savings accumulated 
also were not found. 

Neither Baltimore nor DC showed gains in labor market and savings outcomes. Similarly, females 
demonstrated no gains in these measures. Males in the treatment and treated groups, however, were 
less likely to be employed than their counterparts in the control group. Given their increased probability 
of attending college, this finding likely indicates that Urban Alliance helped male youth enter college, 
specifically four-year colleges, instead of the workforce. However, the impact on males in the treatment 
group was not significant after a multiple-comparison adjustment. None of the three GPA subgroups 
evidenced gains (or losses) in these outcomes. 

TABLE 11 

Employment Impacts 



Mean 



ITT 

TOT 

Outcome 

Treatment 

Control 

Difference 
in means 

Regression 

adjusted 

IV regression 
adjusted 

Probability of having a 
postprogram job 

0.42 

0.49 

-0.066 

-0.066 

-0.133 





(0.042) 

(0.089) 

Postprogram log of wages 

0.63 

0.69 

-0.069 

-0.026 

-0.128 





(0.039) 

(0.182) 

Money accumulated 

392.45 

361.51 

30.944 

28.779 

60.402 





(109.885) 

(230.466) 


Sources: Urban Alliance program data (control variables) and interim outcome survey (outcome variables). 

Notes: IV = instrumental variables. Robust standard errors are given in parentheses. Intent to treat (ITT) compares outcomes of a 
treatment group of individuals who were accepted into the program (but who may or may not have completed the internship) 
with a control group of individuals who were not accepted into the program. Treatment on the treated (TOT) compares outcomes 
of those in the treatment group who completed the internship to those in the control group. The regression-adjusted models 
included the following control measures: program year, gender, neighborhood percentage poverty, previously held a job, and 
junior year GPA. 

* significant at 10%; ** significant at 5%; *** significant at 1% 
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Implications for Practice and Policy 

The Urban Alliance high school internship program strives to change the trajectories of youth who are 
at risk of becoming disconnected, neither attending college nor finding stable employment. It 
intervenes in their lives at a critical juncture— senior year of high school— and offers them training, an 
internship, and mentoring to help them succeed. The youth it targets live in high-poverty areas with 
majority minority populations and low levels of high school graduation and college enrollment. They 
have little exposure to high-skilled employment in their families or neighbors. Many of these youth face 
large financial and social barriers to college, and they are also not adequately prepared for the 
workforce. Within this context, Urban Alliance works to provide youth with the knowledge and 
resources they need to surpass these barriers and enter a path toward future educational attainment 
and financial well-being. In our baseline report (Theodos et al. 2014), we outlined several implications 
and conclusions about the Urban Alliance program based on its implementation. In this report, we focus 
our discussion on conclusions and implications based on our analysis of take-up and impacts. 

Target Population 

As discussed in Theodos et al. (2014), the Urban Alliance program is aimed primarily at the "middle-of- 
the-road" student, that is, students with neither high nor low GPAs. However, the program does not 
enforce eligibility criteria coincident with acceptingonly this targeted population. In our analysis, we 
segmented the sample into three groups and found that the program had significant impacts for youth 
with GPAs in the 2.0 to 3.0 range, the group most representative of the middle-of-the-road target. In 
particular, this group showed a shift from attending two-year colleges to attending four-year colleges 
relative to the control group. 

Low-GPA students in the Urban Alliance program were no more likely to attend college than 
controls. However, the sample size of this group was fairly small, and the point estimates imply that low- 
GPA students may have been more likely to go to four-year colleges than to two-year colleges 
compared with controls, though we cannot state this definitively given the small sample. Given no 
increase in college attendance, one would expect low-GPA students could benefit from the employment 
aspects of the program. However, these students mostly will not be attractive to the employers 
providing internships. Furthermore, they may possess other characteristics that are barriers to 
success— barriers the Urban Alliance program is not designed to deal with. Urban Alliance would be well 
served, as would the youth who apply, if it had firmer eligibility criteria, including a minimum GPA 
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requirement for program entry. Conversely, high-GPA students would be desirable to the employers 
providing internships, but the program may provide little benefit for these students in terms of college 
enrollment beyond what they can achieve for themselves. High-GPA students are already likely to 
attend college, particularly four-year colleges. Those who apply for and participate in the Urban 
Alliance program most likely are students who try to take advantage of any opportunity available, but 
the program may have limited value-added for them. High-GPA students left the Urban Alliance 
program with more comfort filling out the FAFSA and applying for scholarships as well as more comfort 
with soft skills. These increased levels of comfort are important for future success, but they are not 
representative of widespread program impacts. Although Urban Alliance could be more restrictive with 
a maximum GPA, this goal is less easily accomplished than a minimum GPA requirement in terms of 
acceptability to schools and students. In addition, to satisfy some of the high-end employers who 
participate in the program, higher-GPA students may be important to fill slots and keep the employers 
engaged in the program. 

Program Attrition 

Attrition between students’ applications to the Urban Alliance program and their internship 
completions was significant. The attrition occurred over three basic phases: (1) between the application 
and prior to the first day of pre-work, (2) over the course of pre-work, and (3) during the internship. We 
were unable to predict take-up and continuation well by using baseline data, implying that factors we 
could not observe influenced attrition and/or a significant random element was involved. Both 
situations are likely as we could not observe changes in class schedules, extracurricular activities, or 
family issues, all of which are known to affect program participation. Some predictive factors came 
through, but they tell us little. For example, youth who were parents and youth in special education 
were less likely to attend the first session of pre-work. Lower attendance was not surprising for either 
group, but neither group constituted a large contingent of applicants. Having a previous job was a 
positive predictor of attending pre-work, which could indicate that the program attracts youth who 
already have some understanding of the work world. 

Once youth show up for pre-work, their continued participation is under Urban Alliance's influence 
to a greater degree. Some youth will quickly decide the program is not of interest and drop out, while 
others will find that schedule changes, family or personal situations, and other conflicts make it difficult 
to continue. Much of this change in intention likely is random, and the program can always expect some 
attrition. However, it would be worth understanding better the distribution of reasons for dropping out 
of pre-work. Some youth may drop out from lack of interest, but youth who develop scheduling conflicts 
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may wish to continue. If the prevalence of certain reasons could be established, it may be possible for 
Urban Alliance to facilitate participation. For example, if a significant number of students found their 
schedule changed and they could no longer attend the pre-work sessions. Urban Alliance could consider 
a second session at a different time to accommodate those youth. 

Students with lowGPAs who began pre-work were more likely to drop out before completing pre- 
work. Given this group may be outside the appropriate target population, as discussed above, the 
attrition during this phase could be reduced by adhering to stricter eligibility criteria. 

The final stage, the internship, accounted for little of the overall attrition, but a nontrivial number of 
students left this phase before completion. Some left because they secured another job, a good 
outcome, but others were dismissed for performance reasons. Urban Alliance stresses the relationship 
between the program coordinators and the job mentors, as well as other key practices, to minimize the 
likelihood of job dismissal, but the program will likely always have some students who do not perform 
well on their jobs. Onefactor that we found influenced completing the internship was the size of the 
program coordinator’s caseload, with larger caseloads reducing the likelihood of internship completion 
for those youth who began the internship. High caseloads will diminish a program coordinator’s ability 
to work with students and job mentors, leading to possible unchecked behaviors that lead to job 
dismissal. We do not put too much emphasis on this finding, however, because the distribution of 
caseloads was mostly even, with only a few outliers. The potential for individual differences across 
program coordinators may be the explanation, as much as or more than caseload; this possibility is 
something Urban Alliance should further investigate. 

Self-Selection into the Program 

Although high school graduation rates in DC and Baltimore are not high, the graduation rate for 
students who enter their senior year is. Therefore Urban Alliance recruits a group of youth already 
bound to graduate; in fact, 96 percent of the control group graduated. 

Control group youth also took steps to attend college. Over 90 percent of them took the SAT, filled 
out the FAFSA, and applied to college, yet only about 60 percent attended college. Urban Alliance 
appears to have had little impact on improving the college attendance rate overall, though we detected 
positive impacts for certain subgroups of participants. Further, we see that Urban Alliance influenced 
the quality of the college attended, as measured by colleges’ SAT scores for admitted students and 
graduation rates, and there were meaningful gains in college attendance for specific subgroups. 
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Employment is the alternativefor those who do not attend college, yet we found little impact on 
employment. Three-quarters of the sample claimed some work experience prior to beginning the Urban 
Alliance program. Thus most students may have had some preliminary understanding of the basics of 
the world of work, such as showing up at a scheduled time; staying through a shift; and handling 
interactions with coworkers, supervisors, and customers. Furthermore, the high graduation rates imply 
a relatively employable group compared with high school dropouts. However, most youth lacked 
experience at the type of job offered by Urban Alliance, because they either had not held a job 
previously or had worked typical teenage jobs in retail, food service, or child care. Notably, Urban 
Alliance deepened their understandingof the workplace and office-based jobs. This positive effect was 
demonstrated by our finding that the treatment group reported a greater comfort level with job 
applications, hard skills, and soft skills. As recent research has shown, soft skills in particular are critical 
for labor market success. Developing soft skills at this early stage of their careers can set these youth on 
a path that will lead to better opportunities over their lifetime. 

Schools Matter 

Urban Alliance works with high schools in poor neighborhoods, generally drawing students from low- 
income families. However, even within this limited pool of high schools, there was variation in the 
resources and culture of the schools. Because we did not randomize at the high school level, we 
controlled for high school attended by using various statistical techniques. Regardless of the technique, 
once we controlled for the high school each youth attended, several significant differences in outcomes 
became insignificant. We did not have many high school-specific variables to help identify why 
outcomes differed by school. Schools may have had different resources, with some high schools able to 
provide more college-ready activities or more college counselors. Or some high schools may have 
differed in their college attendance culture because of variation in the backgrounds of parents in each 
school’s neighborhood, which may also have resulted in peers reinforcing the idea of a college-bound 
path. It may also be that some schools valued the Urban Alliance program more than others and made 
more accommodations for students to participate. By implication. Urban Alliance may need to focus 
more attention on students from "less supportive” schools. 

Gender Differences 

In general, females are more likely to graduate high school than males and more likely to attend college. 
As a result, it is not surprising that a majority (65 percent) of youth in the Urban Alliance program were 
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female. Our analysis indicated that the Urban Alliance program may benefit males more than females in 
terms of college access. Although high school graduation rates were high for this sample, as discussed 
above, the males in the treatment group showed statistically significantly higher graduation rates than 
males in the control group. They were also more likely to attend college instead of taking jobs, and to 
attend four-year instead of two-year colleges. Although the Urban Alliance program does not address 
the high dropout rate among males, it appears to improve outcomes for male youth who may need a 
little extra support to attain a better future. 

Unexplained Differences 

Our analysis revealed two types of differences that we are unable to explain. One, differences in 
impacts between the two study cohorts, may represent differences in program recruiting or 
implementation between the two years. Our implementation study did not reveal any important 
differences, but it was not designed to detect them. The differences may also be the result of variation 
across students, staff, or job mentors over time. If the difference is the result of this sort of variation, 
then we cannot know which year represents the true impact of the program; additional cohorts would 
need to be studied. 

We also found that program impacts generally accrued to youth in DC, but may not have in 
Baltimore; that is, we were able to detect effects in DC, but not in Baltimore. This result may partially 
reflect lower power in the Baltimore sample, but standardized effect sizes generally confirmed this 
finding. Some differences existed in the application process in the two sites, but we did not find 
differences in the observable characteristics of applicants in the two cities. The length of pre-work 
differed in the two sites, but we have no reason to believe that difference accounted for differences in 
outcomes. Further analysis to investigate differences across the two sites is hampered by small sample 
sizes in Baltimore. Understanding why Baltimore did not show impacts is important, not only for 
improving the program in Baltimore, but to understand howto implement the program in other 
locations. 

Unknown Differences 

Since this study began. Urban Alliance has expanded to Chicago and Northern Virginia. Each site has a 
different race/ethnicity distribution than DC or Baltimore. The sample studied from the latter two cities 
was nearly all African American, while Chicago and Northern Virginia have a more diverse population, 
particularly in terms of including Latino youth. Although we have no specific expectation of differential 
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impacts among different race or ethnic groups, different cultural aspects may need to be taken into 
account in pre-work training, ongoing workshops, mentoring, and matching to job sites. 

Is the Urban Alliance an Employment Program, a College- 
Access Program, or Both? 

The Urban Alliance program serves to give youth an appreciation for what they can attain if they go to 
college, while also teaching hard and soft skills to youth whether or not they attend college. As an 
employment intervention that might prevent disconnection among terminal high school graduates, the 
program has key components considered effective (US Department of Labor et al. 2014); in particular. 
Urban Alliance provides paid employment, soft skills training, mentors, and postprogram support. 
However, as noted above, a minority of program youth did not go on to college, and the youth who 
chose employment over college had previous exposure to the world of work. Further, we did not find 
employment or wage impacts. In our implementation study, we noted some areas for program 
development for aiding in college planning, such as college campus visits. For youth focusing on 
employment, the program may need to consider additional supports for youth not going to college by 
helping facilitate job attainment after the internship is completed or offering more alumni services. 

Conclusion 

Although this evaluation raises questions for Urban Alliance’s consideration, it also demonstrates the 
promise of an internship program for high school youth with comprehensive supports. Research has 
demonstrated that programs offering youth on-the-job learning, paid internships, mentoring, or job 
training alongside mentoring can produce positive impacts; job training on its own has been less 
successful. This study adds to that literature by demonstrating the impacts of an intensive intervention 
for students still in high school but at risk of disconnecting once they graduate that provides a paid job, 
individual support, and job and life skills training. 

Still, there is more to be learned, such as how exclusively to target services, the importance of 
structuring recruitment so as to reduce attrition, why youth from certain schools or program sites may 
not benefit as much from such a program as other youth, and which additional supports might be 
necessary for youth who are not benefitting from a particular program component. Greater 
understanding of these issues would benefit not only Urban Alliance as it continues to refine its 
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program model and implementation, but also other organizations aiming to help at-risk youth transition 
to higher education and positive career pathways. 

The final report resulting from this research, expected in 2017, may further illuminate these areas 
in addition to determining longer-term education and employment outcomes for participating youth. 
This final report will follow youth through potential entry into their third year in college; for youth not 
on a four-year college track, it may detect their graduation from two-year college or their completion of 
a training program and/or entry into the job market. Findings from the final report will help us learn 
more about the long-term trajectories of these youth and others like them. 
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Appendix A, Survey Methodology 

The survey sample included the entire control group and a randomly selected subset of treatment 
group participants that matched the size of the control group (table A.1). 

TABLE A.1 

Survey Sample by Assignment Group and Cohort 


Cohort 

Control 

Treatment 

Total 

2011-12 

186 

186 

372 

2012-13 

176 

176 

352 

Both cohorts 

362 

362 

724 


Source: Urban Alliance high school internship program application forms. 

The survey administration was primarily managed by a subcontractor, SSRS. SSRS e-mailed an 
invitation to each member of the survey sample with a valid e-mail address explaining the purpose of 
the study and the survey. The e-mail highlighted the offer of a $40 gift card for completing the survey 
and invited youth to complete the survey online. SSRS sent one to two follow-up e-mails asking youth to 
complete the survey. After the initial e-mail messages, SSRS mailed letters of invitation to all 
noncompleters that included information on how youth could complete the survey online or by phone. 
SSRS next mailed a second letter of invitation to noncompleters that included a $2 preincentive and the 
offer of a $40 gift card for completing the survey. SSRS sent up to five additional follow-up e-mails to 
youth who had not yet completed the survey. Following the invitation e-mails, SSRS sent two e-mails to 
noncompleters offering a preincentive of a $10 gift card and then a $30 gift card for completing the 
survey. 

SSRS followed up with those who did not complete the survey via the web by telephone. SSRS 
trained call center supervisors and interviewers to administer the survey to ensure accurate data 
collection and maximize response rates. Interviewers received written materials prior to survey 
administration that included an annotated questionnaire, information about the goals of the study, 
pronunciation of key terms, and guidance on overcoming obstacles to accurate answers. 

Before asking whether respondents agreed to the survey, respondents were briefed about the 
confidential and voluntary nature of the survey. If respondents agreed to the survey, the survey 
proceeded. If respondents did not agree, the interviewer or online survey screen thanked them for their 
time and reminded them that they could return to the survey if they changed their mind. The survey 
used slightly different language for the intervention and control groups: the intervention group was told 
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the survey would be evaluating the Urban Alliance High School Internship program, and the control 
group was told the survey was aimed at recent DC and Baltimore high school students. 

SSRS used contact information from Urban Alliance program applications, including phone 
numbers, e-mail addresses, parent/guardian contact information, and an emergency contact. When 
possible Urban Alliance staff provided updated contact information for youth in the treatment group 
with whom they were in contact. SSRS supplemented Urban Alliance’s contact information with 
National Change of Address data, Facebook searches, and contact information from the National 
Student Clearinghouse. 

SSRS called the phone numbers provided by Urban Alliance an average of 16 times if they received 
no answer, a busy tone, or an answering machine, before ending the phone call attempts. SSRS 
contacted nonresponsive numbers at multiple times of the day and varied days of the week. SSRS 
offered respondents the option to schedule a call-back. An SSRS team experienced in refusal 
conversions called youth that refused to complete the survey in an attempt to persuade respondents to 
complete data collection. SSRS staff also called youth who began but did not complete the survey to 
encourage them to complete the survey. After SSRS sent the e-mail messages and letters and made the 
calls, youth who did not respond to the survey received text messages on their cellular phones and 
Facebook messages, when possible. 

The final component of survey administration was performed by a consultant interviewer who was 
highly experienced in field research and locating survey participants. The interviewer attempted to 
make in-person contact with nonresponders at their last known addresses in DC and Baltimore and the 
addresses of their family or emergency contacts. 

Thesurvey for the 2011-12 cohort was open from March 11 through September 27, 2013, and 
from April 7 through August 7, 2014, for the 2012-13 cohort. The goal was to interview youth about 
one year after their predicted high school graduation dates in June. The average time between expected 
high school graduation and survey response was 330 days for the first cohort and 343 days for the 
second cohort. The average time between predicted high school graduation and survey response was 
344 days for the control group and 329 days for the treatment group. 

The survey achieved a 77 percent response rate across assignment groups and cohorts (table A.2). 
The response rate for the first cohort (79 percent) was slightly higher than the second (76 percent). The 
response rate for the treatment group was modestly higher (80 percent) than for the control group (75 
percent). 
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TABLE A.2 


Response Levels and Rates by Treatment Group and Cohort 

Cohort Control Treatment Total 

2011- 12 140(75%) 155(83%) 295(79%) 

2012- 13 130(74%) 136(77%) 266(76%) 

Both cohorts 270(75%) 291(80%) 561(77%) 


Source: Interim outcome survey. 
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Appendix B. Differential Attrition 

Differential attrition between the treatment and control groups was minimal (table B.l). There were a 
few differences between the two groups in the interim survey that were not present at the time of 
program application. Treatment group youth who completed the survey were more likely to have a 
father present in the household (41 versus 30 percent), and a chi-squared test of their living 
arrangements was significant at the .10 level. Control group youth were more likely to be enrolled in 
special education classes (10 versus 5 percent). Other differences, however, were already present at 
baseline: treatment group youth had higher GPAs at the end of their junior year, and control group 
youth were more likely to have a checking or savings account. 

TABLE B.l 

Survey Respondent Characteristics at Baseline, Overall and by Treatment Group 


Characteristic 

All 

Treatment 

Control 

Significance 

Youth demographics 





US citizen 

97% 

97% 

97% 


English language learner 

12% 

12% 

12% 


Female 

67% 

67% 

67% 


Race 

African American 

89% 

87% 

91% 


White 

1% 

1% 

1% 


Hispanic 

6% 

8% 

5% 


Other 

4% 

4% 

3% 


Youth family characteristics 





Mother present in household 

83% 

84% 

83% 


Father present in household 

36% 

41% 

30% 

*** 

Has a child 

3% 

3% 

3% 


Employed adult in household 

77% 

76% 

78% 


Living Arrangement 

Father only 

5% 

6% 

5% 

+ 

Mother only 

53% 

49% 

58% 


Other 

11% 

10% 

12% 


Two parents 

31% 

35% 

25% 


Other characteristics 





Had a previous job 

74% 

75% 

74% 


Has a checking or savings account 

39% 

34% 

45% 

** 

Money saved 

$92 

$74 

$111 
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TABLE B.l CONTINUED 


Characteristic 

All 

Treatment 

Control 

Significance 

Academic achievement and educational attributes 
Number of other schools attended in past three 
years 

0.4 

0.4 

0.5 


In special education 

8% 

5% 

10% 

** 

GPA at end of junior year 

2.7 

2.7 

2.6 

** 

Has taken or plans to take ACT or SAT 

91% 

91% 

91% 


Observations (n) 

561 

291 

270 



Source: Urban Alliance high school internship program application forms. 

Notes: All items had a response rate of 80 percent or more except bank account (70 percent) and "has taken or plans to take ACT 
or SAT" (71 percent). For categorical variables (living arrangement and race), significance of the chi-square test is shown in the 
first category row. 

* significant at 10%; ** significant at 5%; *** significant at 1%; + chi-squared test significant for categorical variable at the 10% 
level 
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Appendix C. Full Sample and Subgroup Impact 
Tables 

TABLE Cl 

Urban Alliance Program Impacts, Full Sample 


Mean ITT TOT 


Outcome (data source) 

Estimation 

technique 

Obs. 

(«) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV 

regression 

adjusted 

Received job help (survey) 

Logit, RE 

553 

0.948 

0.823 

0.124*** 

0.380 

0.128*** 

0.405 

0.261*** 








(0.031) 


(0.058) 

Received college help 
(survey) 

Logit, RE 

555 

0.934 

0.854 

0.079*** 

0.229 

0.081*** 

0.265 

0.168*** 








(0.027) 


(0.054) 

Took SAT (survey) 

Logit, RE 

561 

0.907 

0.911 

-0.004 

-0.014 

-0.015 

-0.035 

-0.038 








(0.022) 


(0.049) 

Took ACT (survey) 

Logit, RE 

561 

0.443 

0.352 

0.091** 

0.184 

0.071* 

0.145 

0.132 








(0.041) 


(0.084) 

Filled out FAFSA (survey) 

Logit, RE 

554 

0.931 

0.921 

0.01 

0.038 

-0.004 

0.000 

-0.008 








(0.021) 


(0.046) 

Comfort with FAFSA and 
scholarships (survey) 

GLS, RE 

547 

3.595 

3.466 

0.129** 

0.200 

0.129** 

0.215 

0.262** 








(0.052) 


(0.109) 

Applied to college (survey) 

Logit, RE 

561 

0.942 

0.919 

0.023 

0.079 

0.01 

0.039 

0.011 








(0.021) 


(0.043) 
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TABLE Cl CONTINUED 


Mean ITT TOT 


Outcome (data source) 

Estimation 

technique 

Obs. 

(n) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV 

regression 

adjusted 

Job application comfort 
(survey) 

GLS, RE 

554 

3.644 

3.584 

0.06 

0.121 

0.06 

0.121 

0.125 








(0.043) 


(0.088) 

Hard skill comfort (survey) 

GLS, RE 

555 

3.664 

3.523 

0.142*** 

0.207 

0.124** 

0.183 

0.254** 








(0.058) 


(0.116) 

Soft skill comfort (survey) 

GLS, RE 

551 

3.736 

3.63 

0.106*** 

0.243 

0.099*** 

0.240 

0.203*** 








(0.035) 


(0.073) 

Graduated high school (HS 
data) 

Logit, RE 

951 

0.98 

0.958 

0.022** 

0.123 

0.012 

0.074 

0.023 








(0.010) 


(0.025) 

Suspended senior year (HS 
data) 

Logit, RE 

968 

0.076 

0.096 

-0.019 

-0.069 

-0.011 

-0.036 

-0.036 








(0.018) 


(0.045) 

Chronically absent senior 
year (HS data) 

Logit, RE 

968 

0.321 

0.312 

0.009 

0.019 

0.048 

0.103 

0.111 








(0.031) 


(0.070) 

Cumulative GPA (HS data) 

GLS, RE 

937 

2.655 

2.555 

0.101** 

0.157 

-0.012 

-0.016 

-0.027 








(0.026) 


(0.060) 

Attended college (NSC) 

Logit, RE 

1059 

0.661 

0.596 

0.065** 

0.125 

0.029 

0.060 

0.062 








(0.033) 


(0.072) 

Attended four-year college 
(NSC) 

Logit, RE 

1059 

0.551 

0.462 

0.089*** 

0.160 

0.035 

0.070 

0.081 








(0.033) 


(0.072) 
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TABLE Cl CONTINUED 


Mean ITT TOT 


Outcome (data source) 

Estimation 

technique 

Obs. 

(/7) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV 

regression 

adjusted 

Attended two-year college 
(NSC) 

Logit, RE 

1059 

0.14 

0.159 

-0.019 

-0.054 

-0.008 

0.000 

-0.009 








(0.023) 


(0.055) 

75th percentile SAT score 
of college attended 
(IPEDS) 

GLS, RE 

328 

1048.1 

1011.8 

36.230” 

0.255 

30.025* 

0.211 

50.648 








(16.391) 


(34.504) 

Retention rate of college 
attended (IPEDS) 

GLS, RE 

590 

65.408 

63.239 

2.169* 

15.591 

0.694 

5.009 

1.515 








(1.106) 


(2.419) 

Graduation rate of college 
attended (IPEDS) 

GLS, RE 

588 

35.972 

32.543 

3.429** 

17.824 

1.506 

7.849 

3.229 








(1.558) 


(3.427) 

Held a postprogram job 
(survey) 

Logit, RE 

561 

0.419 

0.485 

-0.066 

-0.133 

-0.066 

-0.120 

-0.133 








(0.042) 


(0.089) 

Postprogram log of wages 
(survey) 

Logit, RE 

553 

0.626 

0.695 

-0.069 

-0.348 

-0.026 

-0.101 

-0.128 








(0.039) 


(0.182) 

Money accumulated 
(survey) 

GLS, RE 

554 

392.5 

361.5 

30.94 

0.018 

28.779 

0.017 

60.402 








(109.885) 


(230.466) 


Sources: Urban Alliance program data, interim outcome survey, DC Public Schools, Baltimore Public School Board, DC Public Charter School Board, individual charter schools in DC, 
National Student Clearinghouse (NSC), and Integrated Postsecondary Education Data System (IPEDS). 

Notes: Obs. = observations; RE = random effects; GLS = generalized least squares; IV = instrumental variables; HS = high school. Robust standard errors are given in parentheses. 
Intent to treat (ITT) compares outcomes of a treatment group of individuals who were accepted into the program (but who mayor may not have completed the internship) with a 
control group of individuals who were not accepted into the program. T reatment on the treated (TOT) compares outcomes of those in the treatment group who completed the 
internship to those in the control group. The regression-adjusted models included the following control measures; program year, gender, neighborhood percentage poverty, 
previously held a job, and junior year GPA. 

* significant at 10%; ** significant at 5%; *** significant at 1% 
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TABLE C.2 


Urban Alliance Program Impacts, Washington, DC 


Mean ITT TOT 


Outcome (data source) 

Estimation 

technique 

Obs. 

[n) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV regression 
adjusted 

Received job help (survey) 

Logit, RE 

464 

0.948 

0.822 

0.126*** 

0.383 

0.134*** 

0.415 

0.274*** 








(0.033) 


(0.063) 

Received college help 
(survey) 

Logit, RE 

466 

0.928 

0.87 

0.058** 

0.167 

0.065** 

0.201 

0.128** 








(0.030) 


(0.058) 

Took SAT (survey) 

Logit, RE 

471 

0.913 

0.908 

0.006 

0.021 

-0.01 

0.000 

-0.023 








(0.025) 


(0.053) 

Took ACT (survey) 

Logit, RE 

471 

0.48 

0.41 

0.07 

0.141 

0.052 

0.100 

0.1 








(0.046) 


(0.095) 

Filled out FAFSA (survey) 

Logit, RE 

466 

0.925 

0.921 

0.004 

0.015 

-0.014 

0.000 

-0.031 








(0.025) 


(0.052) 

Comfort with FAFSA and 
scholarships (survey) 

GLS, RE 

459 

3.606 

3.507 

0.099* 

0.155 

0.102* 

0.172 

0.208* 








(0.056) 


(0.116) 

Applied to college (survey) 

Logit, RE 

471 

0.953 

0.931 

0.022 

0.086 

0.001 

0.004 

0 








(0.022) 


(0.044) 

Job application comfort 
(survey) 

GLS, RE 

465 

3.657 

3.574 

0.083* 

0.163 

0.090* 

0.183 

0.186* 








(0.046) 


(0.096) 

Hard skill comfort (survey) 

GLS, RE 

465 

3.671 

3.559 

0.112* 

0.169 

0.102* 

0.154 

0.194 








(0.062) 


(0.125) 

Soft skill comfort (survey) 

GLS, RE 

461 

3.738 

3.622 

0.116*** 

0.260 

0.110*** 

0.260 

0.224*** 








(0.040) 


(0.082) 
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TABLE C.2 CONTINUED 





Mean 


ITT 


TOT 

Outcome (data source) 

Estimation 

technique 

Obs. 

[n) 

Treatment Control 

Difference 
in means 

Effect Regression 

size adjusted 

Effect 

size 

IV regression 
adjusted 


Graduated high school (HS 
data) 

Logit, RE 

781 

0.977 

0.956 

0.022* 

0.118 

0.019 

0.059 

0.01 








(0.017) 


(0.029) 

Suspended senior year (HS 
data) 

Logit, RE 

797 

0.079 

0.111 

-0.032 

-0.112 

-0.02 

-0.070 

-0.064 








(0.021) 


(0.052) 

Chronically absent senior 
year (HS data) 

Logit, RE 

797 

0.349 

0.352 

-0.003 

-0.006 

0.05 

0.105 

0.122 








(0.037) 


(0.082) 

Cumulative GPA (HS data) 

GLS, RE 

766 

2.735 

2.6 

0.135*** 

0.211 

0.01 

0.016 

0.024 








(0.026) 


(0.061) 

Attended college (NSC) 

Logit, RE 

859 

0.694 

0.622 

0.071** 

0.149 

0.041 

0.085 

0.081 








(0.035) 


(0.077) 

Attended four-year college 
(NSC) 

Logit, RE 

859 

0.628 

0.54 

0.089** 

0.163 

0.053 

0.102 

0.116 








(0.035) 


(0.082) 

Attended two-year college 
(NSC) 

Logit, RE 

859 

0.096 

0.104 

-0.008 

-0.027 

-0.002 

-0.007 

-0.002 








(0.021) 


(0.054) 

75th percentile SAT score 
of college attended (IPEDS) 

GLS, RE 

295 

1050.5 

1007.6 

42.829** 

0.294 

33.794* 

0.232 

58.602 








(17.921) 


(36.543) 

Retention rate of college 
attended (IPEDS) 

GLS, RE 

519 

66.612 

64.386 

2.226* 

17.424 

1.639 

12.794 

3.543 








(1.128) 


(2.400) 
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TABLE C.2 CONTINUED 


Mean 


ITT 


TOT 


Outcome (data source) 

Estimation 

technique 

Obs. 

(n) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV regression 
adjusted 

Graduation rate of college 
attended (IPEDS) 

GLS, RE 

520 

37.7 

34.324 

3.376** 

18.624 

2.658 

14.645 

5.741 








(1.622) 


(3.496) 

Held a postprogram job 
(survey) 

Logit, RE 

471 

0.413 

0.456 

-0.043 

-0.087 

-0.045 

-0.091 

-0.097 








(0.046) 


(0.098) 

Postprogram log of wages 
(survey) 

Logit, RE 

464 

0.609 

0.611 

-0.003 

-0.015 

0.003 

0.015 

0 








(0.042) 


(0.197) 

Money accumulated 
(survey) 

GLS, RE 

465 

392.3 

312.2 

80.12 

0.051 

68.316 

0.044 

118.486 








(112.413) 


(234.124) 


Sources: Urban Alliance program data, interim outcome survey, DC Public Schools, Baltimore Public School Board, DC Public Charter School Board, individual charter schools in DC, 
National Student Clearinghouse (NSC), and Integrated Postsecondary Education Data System (IPEDS). 

Notes: Ohs. = observations; RE = random effects; GLS = generalized least squares; IV = instrumental variables; HS = high school. Robust standard errors are given in parentheses. 
Intent to treat (ITT) compares outcomes of a treatment group of individuals who were accepted into the program (but who mayor may not have completed the internship) with a 
control group of individuals who were not accepted into the program. Treatment on the treated (TOT) compares outcomes of those in the treatment group who completed the 
internship to those in the control group. The regression-adjusted models included the following control measures; program year, gender, neighborhood percentage poverty, 
previously held a job, and junior year G PA. 

* significant at 10%; ** significant at 5%; *** significant at 1% 
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TABLE C.3 


Urban Alliance Program Impacts, Baltimore 


Mean ITT TOT 


Outcome (data source) 

Estimation 

technique 

Obs. 

(/7) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV 

regression 

adjusted 

Received job help (survey) 

Logit, RE 

89 

0.946 

0.827 

0.119* 

0.360 

- 


0.287** 








- 


(0.144) 

Received college help (survey) 

Logit, RE 

89 

0.973 

0.788 

0.185** 

0.539 

- 


0.436*** 








- 


(0.160) 

Took SAT (survey) 

Logit, RE 

90 

0.865 

0.925 

-0.06 

-0.199 

-0.317 

-1.028 

-0.051 








(0.331) 


(0.123) 

Took ACT (survey) 

Logit, RE 

90 

0.189 

0.113 

0.076 

0.215 

0.061 

0.173 

0.082 








(0.076) 


(0.154) 

Filled out FAFSA (survey) 

Logit, RE 

88 

0.973 

0.922 

0.051 

0.219 

0 

0.000 

0.051 








0.000 


(0.092) 

Comfort with FAFSA and 
scholarships (survey) 

GLS, RE 

88 

3.514 

3.298 

0.216 

0.321 

0.229 

0.340 

0.442 








(0.163) 


(0.312) 

Applied to college (survey) 

Logit, RE 

90 

0.865 

0.868 

-0.003 

-0.009 

-0.038 

-0.088 

-0.045 








(0.132) 


(0.141) 

Job application comfort (survey) 

GLS, RE 

89 

3.554 

3.625 

-0.071 

-0.088 

-0.091 

-0.113 

-0.18 








(0.129) 


(0.260) 

Hard skill comfort (survey) 

GLS, RE 

90 

3.622 

3.377 

0.244 

0.458 

0.387** 

0.727 

0.792** 








(0.175) 


(0.316) 

Soft skill comfort (survey) 

GLS, RE 

90 

3.724 

3.664 

0.06 

0.171 

0.107 

0.305 

0.23 








(0.084) 


(0.164) 

Graduated high school (HS data) 

Logit, RE 

170 

0.991 

0.967 

0.024 

0.182 

- 


0.063 








- 


(0.049) 
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TABLE C.3 CONTINUED 


Mean jTT TOT 

IV 


Outcome (data source) 

Estimation 

technique 

Obs. 

(«) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

regression 

adjusted 

Suspended senior year (HS data) 

Logit, RE 

171 

0.064 

0.033 

0.031 

0.138 

- 


0.037 








- 


(0.082) 

Chronically absent senior year 
(HS data) 

Logit, RE 

171 

0.182 

0.148 

0.034 

0.090 

0.027 

0.072 

0.077 








(0.046) 


(0.125) 

Cumulative GPA (HS data) 

GLS, RE 

171 

2.273 

2.38 

-0.107 

-0.158 

-0.074 

-0.111 

-0.13 








(0.074) 


(0.166) 

Attended college (NSC) 

Logit, RE 

200 

0.504 

0.506 

-0.002 

-0.004 

-0.008 

-0.016 

-0.014 








(0.072) 


(0.195) 

Attended four-year college 
(NSC) 

Logit, RE 

200 

0.176 

0.198 

-0.021 

-0.051 

-0.033 

-0.077 

-0.085 








(0.055) 


(0.156) 

Attended two-year college (NSC) 

Logit, RE 

200 

0.353 

0.346 

0.007 

0.015 

0.002 

0.004 

0.015 








(0.070) 


(0.183) 

75th percentile SAT score of 
college attended (IPEDS) 

GLS, RE 

33 

1018.2 

1034.1 

-15.827 

-0.144 

-4.821 

-0.044 

-12.852 








(45.136) 


(188.147) 

Retention rate of college 
attended (IPEDS) 

GLS, RE 

71 

54.805 

57.2 

-2.395 

13.311 

-2.531 

-14.091 

-15.812 








(4.665) 


(17.286) 

Graduation rate of college 
attended (IPEDS) 

GLS, RE 

68 

20.374 

22.428 

-2.055 

-9.405 

-1.415 

-6.469 

-19.108 








(5.517) 


(21.512) 

Held a postprogram job (survey) 

Logit, RE 

90 

0.459 

0.604 

-0.144 

-0.280 

-0.111 

-0.220 

-0.18 


(0.107) (0.228) 
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TABLE C.3 CONTINUED 





Mean 




ITT 


TOT 

Outcome (data source) 

Estimation 

technique 

Obs. 

(/i) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV 

regression 

adjusted 

Postprogram log of wages 
(survey) 

Logit, RE 

89 

0.743 

1.038 

-0.295 

-1.698 

-0.134 

-0.761 

-0.586 








(0.111) 


(0.499) 

Money accumulated (survey) 

GLS, RE 

89 

393.3 

564.4 

-171.10 

-0.075 

-189.976 

-0.082 

-403.445 


(399.065) (802.602) 


Sources: Urban Alliance program data, interim outcome survey, DC Public Schools, Baltimore Public School Board, DC Public Charter School Board, individual charter schools in DC, 
National Student Clearinghouse (NSC), and Integrated Postsecondary Education Data System (IPEDS). 

Notes: Ohs. = observations; RE = random effects; GLS = generalized least squares; IV = instrumental variables; HS = high school. " indicates insufficient variation in the dependent 
variable to estimate. Robust standard errors are given in parentheses. Intent to treat (ITT) compares outcomes of a treatment group of individuals who were accepted into the 
program (but who mayor may not have completed the internship) with a control group of individuals who were not accepted into the program. Treatment on the treated (TOT) 
compares outcomes of those in the treatment group who completed the internship to those in the control group. The regression-adjusted models included the following control 
measures; program year, gender, neighborhood percentage poverty, previously held a job, and junior year GPA. 

* significant at 10%; ** significant at 5%; *** significant at 1% 
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TABLE C.4 


Urban Alliance Program Impacts, Females 





Mean 


ITT 


TOT 

Outcome (data source) 

Estimation 

technique 

Obs. 

in) 

Treatment Control 

Difference 
in means 

Effect Regression 

size adjusted 

Effect 

size 

IV 

regression 

adjusted 


Received job help (survey) 

Logit, RE 

372 

0.948 

0.844 

0.105*** 

0.346 

0.113*** 

0.373 

0.234*** 








(0.035) 


(0.068) 

Received college help 
(survey) 

Logit, RE 

373 

0.948 

0.878 

0.070** 

0.250 

0.079** 

0.282 

0.154** 








(0.031) 


(0.061) 

Took SAT (survey) 

Logit, RE 

376 

0.908 

0.917 

-0.009 

0.000 

-0.012 

-0.035 

-0.033 








(0.028) 


(0.060) 

Took ACT (survey) 

Logit, RE 

376 

0.477 

0.387 

0.090* 

0.181 

0.093* 

0.187 

0.16 








(0.051) 


(0.103) 

Filled out FAFSA (survey) 

Logit, RE 

374 

0.943 

0.939 

0.004 

0.017 

-0.001 

-0.004 

-0.002 








(0.021) 


(0.050) 

Comfort with FAFSA and 
scholarships (survey) 

GLS, RE 

366 

3.599 

3.471 

0.128** 

0.218 

0.119* 

0.203 

0.242* 








(0.062) 


(0.130) 

Applied to college (survey) 

Logit, RE 

376 

0.933 

0.934 

0 

0.000 

0.011 

0.044 

-0.006 








(0.035) 


(0.051) 

Job application comfort 
(survey) 

GLS, RE 

371 

3.626 

3.556 

0.069 

0.135 

0.066 

0.129 

0.134 








(0.054) 


(0.112) 

Hard skill comfort (survey) 

GLS, RE 

372 

3.71 

3.514 

0.196*** 

0.297 

0.172** 

0.260 

0.358** 


Soft skill comfort (survey) 

GLS, RE 

369 

3.723 

3.62 

0.102** 

0.244 

0.099** 

0.236 

0.205** 








(0.044) 


(0.091) 
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TABLE C.4 CONTINUED 


Mean ITT TOT 


Outcome (data source) 

Estimation 

technique 

Obs. 

{n) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV 

regression 

adjusted 

Graduated high school (HS 
data) 

Logit, RE 

625 

0.977 

0.969 

0.008 

0.051 

0 

0.000 

-0.004 








(0.017) 


(0.029) 

Suspended senior year (HS 
data) 

Logit, RE 

633 

0.066 

0.072 

-0.006 

-0.024 

-0.005 

-0.020 

-0.018 








(0.021) 


(0.051) 

Chronically absent senior 
year (HS data) 

Logit, RE 

633 

0.363 

0.349 

0.014 

0.029 

0.046 

0.096 

0.101 








(0.039) 


(0.087) 

Cumulative GPA (HS data) 

GLS, RE 

620 

2.691 

2.633 

0.058 

0.091 

-0.017 

-0.016 

-0.047 








(0.032) 


(0.072) 

Attended college (NSC) 

Logit, RE 

685 

0.671 

0.655 

0.016 

0.034 

-0.011 

-0.021 

-0.022 








(0.040) 


(0.088) 

Attended four-year 
college (NSC) 

Logit, RE 

685 

0.571 

0.527 

0.044 

0.088 

-0.009 

-0.018 

-0.009 








(0.043) 


(0.088) 

Attended two-year college 
(NSC) 

Logit, RE 

685 

0.129 

0.159 

-0.031 

-0.087 

-0.011 

-0.029 

-0.015 








(0.027) 


(0.065) 

75th percentile SAT score 
of college attended 
(IPEDS) 

GLS, RE 

215 

1041.9 

1013.6 

28.305 

0.214 

20.157 

0.153 

34.354 








(18.349) 


(39.206) 

Retention rate of college 
attended (IPEDS) 

GLS, RE 

402 

65.315 

63.697 

1.618 

12.114 

-0.376 

-2.770 

-0.584 








(1.279) 


(2.787) 
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TABLE C.4 CONTINUED 


Mean ITT TOT 


Outcome (data source) 

Estimation 

technique 

Obs. 

{n) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV 

regression 

adjusted 

Graduation rate of college 
attended (IPEDS) 

GLS, RE 

400 

36.303 

33.157 

3.146 

17.449 

0.472 

2.618 

1.255 








(1.747) 


(3.784) 

Held a postprogram job 
(survey) 

Logit, RE 

376 

0.405 

0.436 

-0.031 

-0.061 

-0.032 

-0.061 

-0.069 








(0.051) 


(0.109) 

Postprogram log of wages 
(survey) 

Logit, RE 

372 

0.634 

0.636 

-0.003 

-0.016 

-0.006 

-0.033 

-0.043 








(0.048) 


(0.220) 

Money accumulated 
(survey) 

GLS, RE 

372 

412.2 

303.8 

108.40 

0.066 

89.541 

0.055 

183.606 








(128.976) 


(273.040) 


Sources: Urban Alliance program data, interim outcome survey, DC Public Schools, Baltimore Public School Board, DC Public Charter School Board, individual charter schools in DC, 
National Student Clearinghouse (NSC), and Integrated Postsecondary Education Data System (IPEDS). 

Notes: Obs. = observations; RE = random effects; GLS = generalized least squares; IV = instrumental variables; HS = high school. Robust standard errors are given in parentheses. 
Intent to treat (ITT) compares outcomes of a treatment group of individuals who were accepted into the program (but who mayor may not have completed the internship) with a 
control group of individuals who were not accepted into the program. Treatment on the treated (TOT) compares outcomes of those in the treatment group who completed the 
internship to those in the control group. The regression-adjusted models included the following control measures; program year, gender, neighborhood percentage poverty, 
previously held a job, and junior year G PA. 

* significant at 10%; ** significant at 5%; *** significant at 1% 
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TABLE C.5 


Urban Alliance Program Impacts, Males 


Mean ITT TOT 


Outcome (data source) 

Estimation 

technique 

Obs. 

(/7) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV 

regression 

adjusted 

Received job help (survey) 

Logit, RE 

181 

0.947 

0.782 

0.165*” 

0.485 

0.187** 

0.550 

0.370*** 








(0.074) 


(0.107) 

Received college help 
(survey) 

Logit, RE 

182 

0.904 

0.807 

0.097* 

0.276 

0.091* 

0.259 

0.185* 








(0.054) 


(0.112) 

Took SAT (survey) 

Logit, RE 

185 

0.906 

0.899 

0.007 

0.024 

-0.025 

-0.067 

-0.041 








(0.037) 


(0.086) 

Took ACT (survey) 

Logit, RE 

185 

0.375 

0.281 

0.094 

0.199 

0.071 

0.151 

0.171 








(0.071) 


(0.147) 

Filled out FAFSA (survey) 

Logit, RE 

180 

0.905 

0.882 

0.023 

0.075 

0.006 

0.019 

0.001 








(0.047) 


(0.099) 

Comfort with FAFSA and 
scholarships (survey) 

GLS, RE 

181 

3.585 

3.454 

0.131 

0.209 

0.153 

0.244 

0.309 








(0.099) 


(0.204) 

Applied to college (survey) 

Logit, RE 

185 

0.958 

0.888 

0.071* 

0.268 

0.056 

0.211 

0.085 








(0.043) 


(0.081) 

Job application comfort 
(survey) 

GLS, RE 

183 

3.68 

3.641 

0.039 

0.055 

0.059 

0.083 

0.12 








(0.071) 


(0.144) 

Hard skill comfort 
(survey) 

GLS, RE 

183 

3.573 

3.54 

0.033 

0.071 

0.052 

0.112 

0.158 








(0.108) 


(0.212) 

Soft skill comfort (survey) 

GLS, RE 

182 

3.763 

3.651 

0.111* 

0.278 

0.098 

0.246 

0.195 








(0.061) 


(0.123) 
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TABLE C.5 CONTINUED 


Mean ITT TOT 

IV 


Outcome (data source) 

Estimation 

technique 

Obs. 

(n) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

regression 

adjusted 

Graduated high school 
(HS data) 

Logit, RE 

326 

0.986 

0.94 

0.045" 

0.261 

0.040* 

0.232 

0.079 








(0.024) 


(0.048) 

Suspended senior year 
(HS data) 

Logit, RE 

335 

0.097 

0.134 

-0.037 

-0.096 

-0.022 

-0.064 

-0.034 








(0.036) 


(0.089) 

Chronically absent senior 
year (HS data) 

Logit, RE 

335 

0.236 

0.252 

-0.016 

-0.023 

0.028 

0.065 

0.062 








(0.049) 


(0.115) 

Cumulative GPA (HS data) 

GLS, RE 

317 

2.581 

2.418 

0.163" 

0.261 

0.005 

0.008 

0.027 








(0.043) 


(0.097) 

Attended college (NSC) 

Logit, RE 

374 

0.643 

0.496 

0.147"* 

0.299 

0.116** 

0.236 

0.264** 








(0.053) 


(0.129) 

Attended four-year 
college (NSC) 

Logit, RE 

374 

0.515 

0.353 

0.161*** 

0.323 

0.119" 

0.239 

0.267** 








(0.053) 


(0.128) 

Attended two-year 
college (NSC) 

Logit, RE 

374 

0.162 

0.158 

0.004 

0.011 

0.01 

0.027 

0.042 








(0.039) 


(0.104) 

75th percentile SAT score 
of college attended 
(IPEDS) 

GLS, RE 

113 

1058.9 

1007.7 

51.13 

0.320 

44.204 

0.276 

126.943 








(33.499) 


(85.105) 

Retention rate of college 
attended (IPEDS) 

GLS, RE 

188 

65.598 

62.161 

3.438 

23.077 

2.177 

14.612 

5.825 
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TABLE C.5 CONTINUED 


Mean ITT TOT 


Outcome (data source) 

Estimation 

technique 

Obs. 

(n) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV 

regression 

adjusted 

Graduation rate of college 
attended (IPEDS) 

GLS, RE 

188 

35.29 

31.143 

4.147 

19.314 

3.381 

15.746 

8.53 








(3.245) 


(7.502) 

Held a postprogram job 
(survey) 

Logit, RE 

185 

0.448 

0.584 

-0.136* 

-0.259 

-0.165** 

-0.319 

-0.302** 








(0.079) 


(0.154) 

Postprogram log of wages 
(survey) 

Logit, RE 

181 

0.61 

0.816 

-0.206 

-0.906 

-0.078 

-0.317 

-0.439 








(0.070) 


(0.329) 

Money accumulated 
(survey) 

GLS, RE 

182 

352.4 

480.3 

-127.92 

-0.070 

-133.069 

-0.074 

-256.245 








(207.808) 


(406.293) 


Sources: Urban Alliance program data, interim outcome survey, DC Public Schools, Baltimore Public School Board, DC Public Charter School Board, individual charter schools in DC, 
National Student Clearinghouse (NSC), and Integrated Postsecondary Education Data System (IPEDS). 

Notes: Ohs. = observations; RE = random effects; GLS = generalized least squares; IV = instrumental variables; HS = high school. Robust standard errors are given in parentheses. 
Intent to treat (ITT) compares outcomes of a treatment group of individuals who were accepted into the program (but who mayor may not have completed the internship) with a 
control group of individuals who were not accepted into the program. Treatment on the treated (TOT) compares outcomes of those in the treatment group who completed the 
internship to those in the control group. The regression-adjusted models included the following control measures; program year, gender, neighborhood percentage poverty, 
previously held a job, and junior year G PA. 

* significant at 10%; ** significant at 5%; *** significant at 1% 
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TABLE C.6 


Urban Alliance Program Impacts, GPA 0.0 to 2.0 


Mean ITT TOT 


Outcome (data source) 

Estimation 

technique 

Obs. 

(/7) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV 

regression 

adjusted 

Received job help (survey) 

Logit, RE 

75 

0.939 

0.857 

0.082 

0.264 

0.096 

0.309 

0.235 








(0.089) 


(0.242) 

Received college help 
(survey) 

Logit, RE 

75 

0.906 

0.791 

0.116 

0.314 

0.174** 

0.471 

0.582** 








(0.082) 


(0.290) 

Took SAT (survey) 

Logit, RE 

77 

0.765 

0.721 

0.044 

0.100 

0.056 

0.127 

0.159 








(0.110) 


(0.303) 

Took ACT (survey) 

Logit, RE 

77 

0.353 

0.233 

0.12 

0.264 

0.152 

0.334 

0.268 








(0.103) 


(0.328) 

Filled out FAFSA (survey) 

Logit, RE 

74 

0.781 

0.81 

-0.028 

-0.049 

0.004 

0.010 

-0.053 








(0.086) 


(0.313) 

Comfort with FAFSA and 
scholarships (survey) 

GLS, RE 

74 

3.545 

3.573 

-0.028 

-0.033 

0.049 

0.080 

-0.019 








(0.156) 


(0.432) 

Applied to college (survey) 

Logit, RE 

77 

0.735 

0.767 

-0.032 

-0.069 

0.023 

0.053 

0.009 








(0.101) 


(0.299) 

Job application comfort 
(survey) 

GLS, RE 

74 

3.591 

3.756 

-0.165 

-0.303 

-0.131 

-0.246 

-0.37 








(0.129) 


(0.394) 

Hard skill comfort (survey) 

GLS, RE 

75 

3.618 

3.634 

-0.016 

-0.017 

-0.013 

-0.017 

-0.036 








(0.145) 


(0.423) 

Soft skill comfort (survey) 

GLS, RE 

73 

3.681 

3.707 

-0.026 

-0.048 

-0.08 

-0.191 

-0.187 








(0.091) 


(0.248) 
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TABLE C.6 CONTINUED 


Mean ITT TOT 


IV 

Estimation Obs. Difference Effect Regression Effect regression 

Outcome (data source) technique (n) Treatment Control in means size adjusted size adjusted 


Graduated high school (HS 
data) 

Logit, RE 

131 

0.888 

0.902 

-0.014 

-0.032 

0.007 

0.023 

0.033 








(0.056) 


(0.233) 

Suspended senior year 
(HS data) 

Logit, RE 

140 

0.14 

0.13 

0.01 

0.029 

0.036 

0.105 

0.014 








(0.060) 


(0.253) 

Chronically absent senior 
year (HS data) 

Logit, RE 

140 

0.581 

0.63 

-0.048 

-0.081 

-0.031 

-0.061 

-0.304 








(0.092) 


(0.347) 

Cumulative GPA (HS data) 

GLS, RE 

135 

1.731 

1.701 

0.03 

0.085 

0.07 

0.198 

0.299 








(0.059) 


(0.254) 

Attended college (NSC) 

Logit, RE 

149 

0.37 

0.368 

0.001 

0.002 

0.011 

0.023 

0.027 








(0.088) 


(0.324) 

Attended four-year 
college (NSC) 

Logit, RE 

149 

0.217 

0.211 

0.007 

0.017 

0.069 

0.167 

0.417 








(0.069) 


(0.263) 

Attended two-year 
college (NSC) 

Logit, RE 

149 

0.163 

0.175 

-0.012 

-0.027 

-0.057 

-0.133 

-0.358 








(0.062) 


(0.252) 

75th percentile SAT score 
of college attended 
(IPEDS) 

GLS, RE 

9 

1036.0 

977.5 

58.5 

0.471 











- 


- 

Retention rate of college 
attended (IPEDS) 

GLS, RE 

47 

59.655 

54.667 

4.989 

38.629 

4.935 

38.211 

23.215 








(4.532) 


(26.848) 
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TABLE C.6 CONTINUED 


Mean ITT TOT 


Outcome (data source) 

Estimation 

technique 

Obs. 

(n) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV 

regression 

adjusted 

Graduation rate of college 
attended (IPEDS) 

GLS, RE 

48 

22.92 

24.657 

-1.737 

-11.89 

0.474 

3.260 

1.754 








(4.735) 


(18.209) 

Held a postprogram job 
(survey) 

Logit, RE 

77 

0.294 

0.419 

-0.124 

-0.248 

-0.087 

-0.165 

-0.23 








(0.111) 


(0.345) 

Postprogram log of wages 
(survey) 

Logit, RE 

75 

0.509 

0.542 

-0.033 

-0.300 

-0.042 

-0.400 

-0.41 








(0.106) 


(0.739) 

Money accumulated 
(survey) 

GLS, RE 

75 

422.7 

279.0 

143.68 

0.055 

-9.288 

-0.004 

-29.346 








(395.796) 


(1251.93) 


Sources: Urban Alliance program data, interim outcome survey, DC Public Schools, Baltimore Public School Board, DC Public Charter School Board, individual charter schools in DC, 
National Student Clearinghouse (NSC), and Integrated Postsecondary Education Data System (IPEDS). 

Notes: Obs. = observations; RE = random effects; GLS = generalized least squares; IV = instrumental variables; HS = high school. " indicates insufficient variation in the dependent 
variable to estimate. Robust standard errors are given in parentheses. Intent to treat (ITT) compares outcomes of a treatment group of individuals who were accepted into the 
program (but who mayor may not have completed the internship) with a control group of individuals who were not accepted into the program. Treatment on the treated (TOT) 
compares outcomes of those in the treatment group who completed the internship to those in the control group. The regression-adjusted models included the following control 
measures; program year, gender, neighborhood percentage poverty, previously held a job, and junior year GPA. 

* significant at 10%; ** significant at 5%; *** significant at 1% 
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TABLE C.7 


Urban Alliance Program Impacts, GPA 2.0 to 3.0 


Mean ITT TOT 


Outcome (data source) 

Estimation 

technique 

Obs. 

(«) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV 

regression 

adjusted 

Received job help (survey) 

Logit, RE 

269 

0.965 

0.803 

0.162*” 

0.514 

0.179*** 

0.568 

0.298*** 








(0.049) 


(0.074) 

Received college help 
(survey) 

Logit, RE 

269 

0.915 

0.866 

0.049 

0.158 

0.041 

0.132 

0.076 








(0.039) 


(0.071) 

Took SAT (survey) 

Logit, RE 

270 

0.916 

0.953 

-0.037 

-0.120 

-0.028 

-0.080 

-0.042 








(0.028) 


(0.054) 

Took ACT (survey) 

Logit, RE 

270 

0.441 

0.362 

0.078 

0.159 

0.088 

0.179 

0.166 








(0.060) 


(0.111) 

Filled out FAFSA (survey) 

Logit, RE 

267 

0.93 

0.944 

-0.013 

-0.041 

-0.009 

-0.037 

-0.015 








(0.030) 


(0.056) 

Comfort with FAFSA and 
scholarships (survey) 

GLS, RE 

263 

3.612 

3.508 

0.103 

0.180 

0.089 

0.155 

0.156 








(0.072) 


(0.133) 

Applied to college (survey) 

Logit, RE 

270 

0.951 

0.961 

-0.01 

-0.048 

-0.008 

-0.039 

-0.021 








(0.024) 


(0.047) 

Job application comfort 
(survey) 

GLS, RE 

267 

3.688 

3.616 

0.072 

0.160 

0.069 

0.153 

0.128 








(0.056) 


(0.103) 

Hard skill comfort 
(survey) 

GLS, RE 

267 

3.695 

3.556 

0.139* 

0.218 

0.107 

0.168 

0.199 








(0.078) 


(0.141) 

Soft skill comfort (survey) 

GLS, RE 

265 

3.746 

3.632 

0.114** 

0.277 

0.106** 

0.257 

0.187** 








(0.051) 


(0.091) 
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TABLE C.7 CONTINUED 


Mean ITT TOT 


Outcome (data source) 

Estimation 

technique 

Obs. 

(«) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV 

regression 

adjusted 

Graduated high school 
(HS data) 

Logit, RE 

488 

0.994 

0.975 

0.019* 

0.172 

0.012 

0.109 

0.036 








(0.012) 


(0.023) 

Suspended senior year 
(HS data) 

Logit, RE 

490 

0.082 

0.093 

-0.01 

-0.036 

-0.007 

-0.025 

-0.018 








(0.025) 


(0.058) 

Chronically absent senior 
year (HS data) 

Logit, RE 

490 

0.326 

0.253 

0.073* 

0.159 

0.099** 

0.215 

0.190** 








(0.044) 


(0.089) 

Cumulative GPA (HS data) 

GLS, RE 

480 

2.538 

2.541 

-0.003 

0.000 

-0.018 

-0.028 

-0.035 








(0.030) 


(0.062) 

Attended college (NSC) 

Logit, RE 

514 

0.672 

0.63 

0.041 

0.086 

0.032 

0.067 

0.058 








(0.048) 


(0.095) 

Attended four-year 
college (NSC) 

Logit, RE 

514 

0.589 

0.462 

0.127*** 

0.255 

0.118** 

0.237 

0.209** 








(0.050) 


(0.095) 

Attended two-year 
college (NSC) 

Logit, RE 

514 

0.114 

0.185 

-0.071** 

-0.203 

-0.075** 

-0.203 

-0.126* 








(0.036) 


(0.068) 

75th percentile SAT score 
of college attended 
(IPEDS) 

GLS, RE 

145 

996.5 

959.6 

36.969 

0.349 

21.42 

0.202 

47.312 








(19.889) 


(36.653) 

Retention rate of college 
attended (IPEDS) 

GLS, RE 

288 

62.221 

61.108 

1.113 

9.156 

0.553 

4.549 

1.12 
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TABLE C.7 CONTINUED 


Mean ITT TOT 


Outcome (data source) 

Estimation 

technique 

Obs. 

(n) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV 

regression 

adjusted 

Graduation rate of college 
attended (IPEDS) 

GLS, RE 

287 

32.454 

28.01 

4.444" 

29.692 

3.666" 

24.494 

7.260" 








(1.822) 


(3.689) 

Held a postprogram job 
(survey) 

Logit, RE 

270 

0.455 

0.496 

-0.042 

-0.080 

-0.024 

-0.040 

-0.021 








(0.061) 


(0.114) 

Postprogram log of wages 
(survey) 

Logit, RE 

269 

0.707 

0.696 

0.011 

0.051 

0.024 

0.110 

0.088 








(0.057) 


(0.241) 

Money accumulated 
(survey) 

GLS, RE 

270 

371.4 

264.1 

107.25 

0.077 

123.561 

0.089 

234.528 








(130.298) 


(241.050) 


Sources: Urban Alliance program data, interim outcome survey, DC Public Schools, Baltimore Public School Board, DC Public Charter School Board, individual charter schools in DC, 
National Student Clearinghouse (NSC), and Integrated Postsecondary Education Data System (IPEDS). 

Notes: Ohs. = observations; RE = random effects; GLS = generalized least squares; IV = instrumental variables; HS = high school. Robust standard errors are given in parentheses. 
Intent to treat (ITT) compares outcomes of a treatment group of individuals who were accepted into the program (but who mayor may not have completed the internship) with a 
control group of individuals who were not accepted into the program. Treatment on the treated (TOT) compares outcomes of those in the treatment group who completed the 
internship to those in the control group. The regression-adjusted models included the following control measures; program year, gender, neighborhood percentage poverty, 
previously held a job, and junior year G PA. 

* significant at 10%; ** significant at 5%; *** significant at 1% 
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TABLE C.8 


Urban Alliance Program Impacts, GPA 3.0 to 4.0 


Mean ITT TOT 


Outcome (data source) 

Estimation 

technique 

Obs. 

(/J) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV 

regression 

adjusted 

Received job help (survey) 

Logit, RE 

160 

0.924 

0.853 

0.071 

0.230 

0.083 

0.268 

0.157* 








(0.051) 


(0.095) 

Received college help 
(survey) 

Logit, RE 

162 

0.957 

0.899 

0.058 

0.230 

0.082* 

0.325 

0.156** 








(0.044) 


(0.076) 

Took SAT (survey) 

Logit, RE 

164 

0.957 

1 

-0.043* 

-0.259 

- 


-0.073 








- 


(0.050) 

Took ACT (survey) 

Logit, RE 

164 

0.5 

0.471 

0.029 

0.058 

0.034 

0.068 

0.045 








(0.085) 


(0.150) 

Filled out FAFSA (survey) 

Logit, RE 

164 

0.979 

0.986 

-0.007 

-0.052 

-0.002 

-0.015 

-0.002 








(0.028) 


(0.042) 

Comfort with FAFSA and 
scholarships (survey) 

GLS, RE 

161 

3.559 

3.418 

0.141 

0.243 

0.161* 

0.277 

0.304* 








(0.095) 


(0.183) 

Applied to college (survey) 

Logit, RE 

164 

1 

0.986 

0.014 

0.179 



0.029 

(0.024) 

Job application comfort 
(survey) 

GLS, RE 

163 

3.596 

3.482 

0.114 

0.224 

0.122 

0.240 

0.232 








(0.084) 


(0.161) 

Hard skill comfort (survey) 

GLS, RE 

163 

3.681 

3.536 

0.145 

0.214 

0.176 

0.260 

0.333 








(0.111) 


(0.212) 

Soft skill comfort (survey) 

GLS, RE 

163 

3.73 

3.643 

0.086 

0.235 

0.104* 

0.285 

0.196* 








(0.059) 


(0.115) 
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TABLE C.8 CONTINUED 


Mean ITT TOT 


Outcome (data source) 

Estimation 

technique 

Obs. 

(/J) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

IV 

regression 

adjusted 

Graduated high school (HS 
data) 

Logit, RE 

277 

1 

0.974 

0.026** 

0.307 

_ 


0.023 








- 


(0.017) 

Suspended senior year (HS 
data) 

Logit, RE 

281 

0.044 

0.065 

-0.021 

-0.092 

-0.021 

-0.092 

-0.041 








(0.029) 


(0.063) 

Chronically absent senior 
year (HS data) 

Logit, RE 

281 

0.206 

0.182 

0.024 

0.060 

0.05 

0.125 

0.131 








(0.054) 


(0.106) 

Cumulative GPA (HS data) 

GLS, RE 

276 

3.276 

3.222 

0.054 

0.121 

0.009 

0.020 

0.018 








(0.052) 


(0.106) 

Attended college (NSC) 

Logit, RE 

300 

0.808 

0.767 

0.041 

0.102 

0.024 

0.060 

0.035 








(0.054) 


(0.102) 

Attended four-year college 
(NSC) 

Logit, RE 

300 

0.692 

0.721 

-0.029 

-0.044 

-0.091 

-0.196 

-0.16 








(0.068) 


(0.114) 

Attended two-year college 
(NSC) 

Logit, RE 

300 

0.159 

0.105 

0.054 

0.154 

0.073 

0.208 

0.168* 








(0.044) 


(0.097) 

75th percentile SAT score 
of college attended 
(IPEDS) 

GLS, RE 

157 

1092.2 

1069.1 

23.121 

0.149 

13.242 

0.085 

18.51 








(28.410) 


(55.077) 

Retention rate of college 
attended (IPEDS) 

GLS, RE 

218 

71.234 

69.933 

1.301 

9.452 

1.03 

7.483 

1.788 








(1.950) 


(3.865) 
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TABLE C.8 CONTINUED 


Mean ITT TOT 


IV 


Outcome (data source) 

Estimation 

technique 

Obs. 

[n) 

Treatment 

Control 

Difference 
in means 

Effect 

size 

Regression 

adjusted 

Effect 

size 

regression 

adjusted 

Graduation rate of college 
attended (IPEDS) 

GLS, RE 

216 

43.496 

43.208 

0.288 

1.354 

-0.477 

-2.210 

-1.969 








(3.259) 


(6.548) 

Held a postprogram job 
(survey) 

Logit, RE 

164 

0.404 

0.486 

-0.081 

-0.161 

-0.106 

-0.201 

-0.198 








(0.076) 


(0.151) 

Postprogram log of wages 
(survey) 

Logit, RE 

160 

0.543 

0.669 

-0.126 

-0.581 

-0.074 

-0.339 

-0.298 








(0.071) 


(0.303) 

Money accumulated 
(survey) 

GLS, RE 

160 

443.1 

454.5 

-11.45 

-0.007 

-32.467 

-0.020 

-60.323 








(213.504) 


(396.972) 


Sources: Urban Alliance program data, interim outcome survey, DC Public Schools, Baltimore Public School Board, DC Public Charter School Board, individual charter schools in DC, 
National Student Clearinghouse (NSC), and Integrated Postsecondary Education Data System (IPEDS). 

Notes: Obs. = observations; RE = random effects; GLS = generalized least squares; IV = instrumental variables; HS = high school. " indicates insufficient variation in the dependent 
variable to estimate. Robust standard errors are given in parentheses. Intent to treat (ITT) compares outcomes of a treatment group of individuals who were accepted into the 
program (but who mayor may not have completed the internship) with a control group of individuals who were not accepted into the program. Treatment on the treated (TOT) 
compares outcomes of those in the treatment group who completed the internship to those in the control group. The regression-adjusted models included the following control 
measures; program year, gender, neighborhood percentage poverty, previously held a job, and junior year GPA. 

* significant at 10%; ** significant at 5%; *** significant at 1% 
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Appendix D, Survey Instrument 

Urban Alliance Evaluation Interim Survey (Cohort 2, Web version) 

The following questions are about how you prepared in high school for work or school. 

(ASK EVERYONE) 

4. Do you have either a high school diploma or GED? 

(Please select only one answer) 

1 High school diploma 

2 GED 

3 Neither 

X Blank/No Answer 

(ASKQ6IFQ4=3) 

6. Are you taking additional courses to obtain your GED? 

1 Yes 

2 No 

X Blank/No Answer 
(ASK EVERYONE) 

7. When you were in high school, did you take any Advanced Placement or IB classes to earn 
credit for college? 

(IB stands for International Baccalaureate) 

1 Yes 

2 No 

8 Don’t Know 
X Blank/No Answer 

(ASK EVERYONE) 

8. When you were in high school, did you take any college classes for credit? 

1 Yes 

2 No 

8 Don’t Know 
X Blank/No Answer 

(ASK EVERYONE) 

9. Did you take either the SAT or ACT? 

1 SAT 

2 ACT 

3 Both 

4 Neither 

X Blank/No Answer 
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(ASKQ11IFQ9=1,3) 

(IFRESPONDENTTOOKTHESAT) 

11. Please enter your total SAT score: 

(PN: ACCEPT ANSWER 0-2400) 

8 Don’t Know 

X Blank/No Answer 

(ASKQ11AIFQ11=8) 
llA. Was your score... 

1 1800 or better 

2 1600 to 1790 

3 1450 to 1590 

4 1250 to 1440 

5 Less than 1250 
8 Don’t Know 

X Blank/No Answer 

(ASKQ12IFQ9=2,3) 

(IFRESPONDENTTOOKTHEACT) 

12. Please enter your composite ACT score: 

(PN: ACCEPT ANSWER 0-36) 

8 Don’t Know 

X Blank/No Answer 

(ASKQ12AIFQ12=8) 

12A. Was your score... 

1 26 or higher 

2 23 to 25 

3 20 to 22 

4 16 to 19 

5 Less than 16 
8 Don’t Know 

X Blank/No Answer 

(ASK EVERYONE) 

13. In order to pay for education after high school did you complete the FAFSA? 
(The FAFSA is the Free Application for Federal Student Aid) 

1 Yes 

2 No 

8 Don’t Know 
X Blank/No Answer 
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(ASK EVERYONE) 

14. In order to pay for education after high school did you apply for grants or scholarships? 

1 Yes 

2 No 

8 Don’t Know 
X Blank/No Answer 

(ASK EVERYONE) 

15. Did you apply to at least one two-year or community college? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKQ16IFQ15=1) 

16. Please enter the number of two-year or community colleges you applied to: 

(PN: ACCEPT ANSWER 1-30) 

X Blank/No Answer 

(PN:ASKQ17IFQ16>0) 

(IF Q16>1, INSERT PARENS) 

(PROVIDE RESPONSE BOXES FOR NUMBER OF COLLEGES IDENTIFIED IN Q16) 

17. What is the name of (each) two-year or community college that you applied to? 

(Please enter response(s) below by providing the full name of the college) 

(Name of two-year or community college) 

8 Don’t Know 
X Blank/No Answer 

(ASK EVERYONE) 

18. Did you apply to at least one four-year college? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKQ19IFQ18=1) 

19. Please enter the number of four-year colleges you applied to: 

(PN: ACCEPT ANSWER 1-30) 

X Blank/No Answer 


APPENDIX D 


87 



(PN:ASKQ20IFQ19>0) 

(IF Q19>1, INSERT PARENS) 

(PROVIDE RESPONSE BOXES FOR NUMBER OF COLLEGES IDENTIFIED IN Q19) 

20. What is the name of (each) four-year college that you applied to? 

(Please enter response below by providing the full name of the college) 

(Name of four-year college) 

8 Don’t Know 
X Blank/No Answer 


(ASK ALL) 

21. Did you apply for military service? 

1 Yes 

2 No 

X Blank/No Answer 

(ASKQ22IFQ21=1) 

22. Did you enlist? 

1 Yes 

2 No 

X Blank/No Answer 

High school students sometimes attend classes or workshops outside of regular school to learn about 
educational opportunities or develop new job skills. These programs might be offered by a high school 
or college, a non-profit such as the Urban Alliance, or a local business. 

(ASK ALL) 

(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-g) SHOULD BE ON THE LEFT) 

23. When you were a high school senior, did you ever attend a class or workshop where you... 

1 Yes 

2 No 

8 Don’t Know 
X Blank/No Answer 

a. Received help choosing a college, such as requesting brochures, writing an admissions essay, 
or applying for admission 

b. Learned about options for paying for college, including completing a FAFSAor applying for 
scholarships 

c. Received career counseling or advice 

d. Learned howto get a job, including creating a resume, writing a cover letter, or completing 
applications 

e. Developed general office skills, such as learning howto use Excel, make photocopies, or file 
papers 

f. Developed communication skills, such as speaking with co-workers, making a presentation, or 
composing a professional email 

g. Learned how to behave on a job, such as how to dress or manage your time 
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(ASKQ24IFQ23A-G=1) 

(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-g) SHOULD BE ON THE LEFT) 

(PN: IFQ23=1 FOR MORE THAN ONE ITEM IN Q23 INSERT SECOND VERBIAGE IN PARENS) 

24. Who offered (the class or workshop you attended/these classes or workshops you attended)? 
Was it... 

1 Yes 

2 No 

8 Don’t Know 
X Blank/No Answer 

a. A high school (This includes your own high school as longas itTsthe classes or workshops were 
outside regular school hours) 

b. A local college 

c. Urban Alliance 

d. Another non-profit organization 

e. A local business 

f. A city or government program 

g. Other (SPECIFY) 

(ASKQ25IFQ23A-G=1) 

(PN: IFQ23=1 FOR MORE THAN ONE ITEM IN Q23 INSERT SECOND VERBIAGE IN PARENS) 

25. About how many total hours did you spend in (this class or workshop/these classes or 
workshops)? Would you say it was... 

(Your best estimate is fine) 

1 Under 10 hours 

2 10 to 25 hours 

3 26 to 50 hours 

4 51 to 75 hours 

5 76 to 100 hours 

6 Over 100 hours 
8 Don’t Know 

X Blank/No Answer 

(ASK ALL) 

(INSERT RESPONSES 5, 6, 10 IF TREATMENT GROUP) 

(PN: ALLOW MULTIPLE RESPONSES, EXCEPT FOR RESPONSE 11) 

(PN:IFQ26=11, ONLY ALLOW SINGLE RESPONSE) 

26. Thinking about all the help you’ve received preparing for your future education, who 
provided this help? Was it... 

(Please select all that apply) 

1 Parent or Foster Parent 

2 Other Relative 

3 Friend or Acquaintance 

4 An employer or co-worker (includes current and former) 

5 (ONLY FOR TREATMENT GROUP) Urban Alliance Program Coordinator ("PC") 

6 (ONLY FOR TREATMENT GROUP) Urban Alliance Alumni Services Staff 

7 Caseworker 


APPENDIX D 


89 



8 T eacher, school counselor, or coach 

9 Clergyperson 

10 (ONLY FORTREATMENT GROUP) Other Urban Alliance Staff 

11 Noone 

12 Other 

X Blank/No Answer 

(ASK ALL) 

(INSERT RESPONSES 5, 6, 10 IF TREATMENT GROUP) 

(PN: ALLOW MULTIPLE RESPONSES EXCEPT FOR RESPONSE 11) 

(PN:IF027=11, ONLY ALLOW SINGLE RESPONSE) 

27. Thinking about all the help you’ve gotten preparing to get and keepa job, who provided this 
help? Was it... 

(Please select all that apply) 

1 Parent or Foster Parent 

2 Other Relative 

3 Friend or Acquaintance 

4 An employer or co-worker (includes current and former) 

5 (ONLY FORTREATMENT GROUP) Urban Alliance Program Coordinator ("PC") 

6 (ONLY FORTREATMENT GROUP) Urban Alliance Alumni Services Staff 

7 Caseworker 

8 T eacher, school counselor, or coach 

9 Clergyperson 

10 (ONLY FORTREATMENT GROUP) Other Urban Alliance Staff 

11 Noone 

12 Other 

X Blank/No Answer 
(ASK Q28-033 IF TREATMENT GROUP) 

28. You applied for the program at Urban Alliance. Did you attend any pre-work training? 

1 Yes 

2 No 

X Blank/No Answer 

(ASKQ29IFQ28=2) 

(PN: ALLOW MULTIPLE RESPONSES) 

29. Why didn't you attend any pre-work training? 

(Please select all that apply) 

1 Class schedule changed 

2 Extra-curricular activities conflicted 

3 No longer interested 

4 Parents wouldn’t let me 

5 Other (SPECIFY) 

8 Don’t Know 

X Blank/No Answer 
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(ASKQ30IFQ28=1) 

30. Did you complete pre-work training? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKQ31IFQ30=2) 

31. Why didn't you complete pre-work training? 

(Please select all that apply) 

1 Class schedule changed 

2 Extra-curricular activities conflicted 

3 No longer interested 

4 Parents wouldn’t let me 

5 Other (SPECIFY) 

8 Don’t Know 

X Blank/No Answer 

(ASKIFTREATMENTGROUP) 

32. Did you complete an internship with Urban Alliance? 

1 Yes 

2 No 

X Blank/No Answer 

(ASKQ33IFQ32=2) 

(PN: ALLOW MULTIPLE RESPONSES) 

(SCRAMBLE 1-9) 

33. Why didn’t you complete your internship with Urban Alliance? 

(Please select all that apply) 

1 Never got assigned to a job 

2 Job was boring 

3 Money wasn’t enough 

4 Change in class schedule 

5 They asked me to leave 

6 Class schedule changed 

7 Extra-curricular activities conflicted 

8 No longer interested 

9 Parents wouldn’t let me 

10 Other (SPECIFY) 

X Blank/No Answer 

(ASK ALL) 

(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-k) SHOULD BE ON THE LEFT) 

34. For each of the following activities, please indicate whether today you would feel very 
comfortable, somewhat comfortable, somewhat uncomfortable, or very uncomfortable in 
completing the activity. 
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1 Very Comfortable 

2 Somewhat Comfortable 

3 Somewhat UNcomfortable 

4 Very UNcomfortable 
8 Don’t Know 

X Blank/No Answer 

a. Identifying grants and scholarships to help pay for college 

b. Completing the FAFSA or scholarship applications 

c. Writing a cover letter or resume 

d. Completing a job application 

e. Asking someone to serve as a job reference 

f. Being interviewed for a job 

g. Performing general office work, such as using Excel, making photocopies, or filing papers 

h. Speaking with adult co-workers and writing professional emails 

i. Making a presentation 

j. Dressing professionally 

k. Completing work assignments on time 

l. Getting to work on time 


Now let’s talk about your current education. 

(ASK ALL) 

35. Since June 2013, have you attended either a two-year or four-year college? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKQ36IFQ35=1) 

36. Please enter the number of two-year or four-year colleges have you attended since June 
2013: 


(PN: ACCEPT ANSWERS 1-10) 

X Blank/No Answer 

(ASK Q37-46, 51, 52 IF Q36=l, 8 OR Q36>1) 

(ASK Q37-46 FOR EACH NUMBER OF COLLEGES SPECIFIED IN Q36) 

(PN: IF 036= 1, INSERT FIRST VERBIAGE; IF Q36>1 OR Q36=8, INSERT SECOND VERBIAGE) 

37. Please enter the name of this college: / Please enter the name of the college you are currently 
attending, or the college you attended most recently: ...Please enter the name of the college 
you attended prior to the last college you mentioned: 

(Please enter response below by providing the full name of the college) 

(Name of two-year or four-year college) 

X Blank/No Answer 

38. Is (INSERT COLLEGE) a four-year or two-year college? 
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1 Four-year college 



2 Two-year college 
8 Don’t Know 
X Blank/No Answer 

(PN: ALLOW BOTH MONTH AND YEAR) 

39. What date did you start attending (INSERT COLLEGE)? Please enter the month and year. 

(PN: ACCEPT 1-12/2013-2014) 

X Blank/No Answer 

40. Are you still attending (INSERT COLLEGE)? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKQ41IFQ40=2) 

(PN: ALLOW BOTH MONTH AND YEAR) 

41. When did you stop? Please enter the month and year. 

(PN: ACCEPT 1-12/2013-2014) 

X Blank/No Answer 

(ASK Q42 IF 040=2) 

(PN: ALLOW ONLY ONE RESPONSE) 

42. What would you say is the main reason that you /eft this college? 

(Please select only one response) 

1 Transferred to a better program or four-year college 

2 Received degree or completed course work 

3 Offered a job 

4 Financial difficulties or couldn’t afford to go 

5 Did not like school or did not get along with other students 

6 Poor grades or failed 

7 Entered military 

8 Moved away from school 

9 Got married 

10 Pregnant, or became the father/mother of a baby 

11 Other child care responsibilities 

12 Other family responsibilities 

13 Personal health or substance problems 

14 Other (Specify) 

X Blank/No Answer 

(IFQ40=1, INSERT FIRST VERBIAGE IN PARENS; IFQ40=2, INSERT SECOND VERBIAGE IN PARENS) 

43. (Are/were) you a full-time or part-time student? 

1 Full-time student 

2 Part-time student 

X Blank/No Answer 
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(IFQ40=1, INSERT FIRST VERBIAGE IN PARENS; IFQ40=2, INSERT SECOND VERBIAGE IN PARENS) 

44. How many credits (have you earned/did you earn) at this school? Include credits applied from 
high school and credits from all complete courses . Your best estimate is fine. 

(Please enter response below) 

(PN: ACCEPT ANSWERS 0-200) 

1 I (have/did) notcomplete(d) a full semester yet 
8 Don’t Know 

X Blank/No Answer 

(IFQ40=1, INSERT FIRST VERBIAGE IN PARENS; IFQ40=2, INSERT SECOND VERBIAGE IN PARENS) 

45. Please enter how many credits (are/were) needed to graduate. Your best estimate is fine. 

(PN: ACCEPT ANSWERS 0-200) 

2 Not applicable 

8 Don’t Know 

X Blank/No Answer 

(IF 040=1, INSERT FIRST VERBIAGE IN PARENS; IFQ40=2, INSERT SECOND VERBIAGE IN PARENS) 
(PN:ALLOWONE DECIMAL) 

46. What (is/was) your total CPA across all terms at (INSERT COLLEGE)? Your best estimate is 
fine. 

(Please enter response below) 

(PN: ACCEPT ANSWERS 0-4.50) 

1 I (have/did) notcomplete(d) a full semester yet 
8 Don’t Know 
X Blank/No Answer 

(ASK Q51 IN SEQUENCE IF Q38=2) 

51. Have you received a certificate, license, or degree from (INSERT TWO-YEAR COLLEGE)? 

1 Yes 

2 No 

X Blank/No Answer 

(ASKQ52IFQ51=1) 

(PN: ALLOW MULTIPLE RESPONSES) 

52. Please enter the types of certificates, licenses, or degrees you have received: 

(Specify) 

(Specify) 

(Specify) 

8 Don’t Know 
X Blank/No Answer 

(ASKQ47IFQ35=1) 

47. Since you began college, have you ever taken a remedial, adult basic education (ABE), or 
developmental learning course? These courses help students develop basic reading, writing, 
and mathematic skills to be successful in college. 
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1 Yes 

2 No 

X Blank/No Answer 
(ASKQ48IFQ47=1) 

48. Please enter the number of remedial, adult basic education (ABE) or developmental learning 
courses you have taken: 

(PN: ACCEPT ASNWER 1-50) 

X Blank/No Answer 

(ASKQ49IFQ40=1) 

(INSERT Q49C ONLY IF RESPONDENT ATTENDED HIGH SCHOOL IN DC) 

(PN: INSERT FIRST RESPONSE IN Q37,THE NAME OF THE MOST RECENT COLLEGE) 

(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-h) SHOULD BE ON THE LEFT) 

49. The following is a list of ways that people pay for school. For each one, tell me if it pays for all 
of your expenses, most of your expenses, some of your expenses, or none of your expenses 
while attending (INSERT FIRST RESPONSE IN 037). 

1 All of your expenses 

2 Most of your expenses 

3 Some of your expenses 

4 None of your expenses 

8 Don’t Know 

X Blank/No Answer 

a. Parents/spouse/relatives 

b. Work-study program 

c. TAG/LEAP (DC ONLY) 

d. Other Grants/scholarships 

e. Student loans, other loans (e.g., bank) 

f. Employer contribution program 

g. Public assistance (e.g., welfare, unemployment) 

h. Other personal income/savings 

(ASK Q50 IF Q38=2 FOR ANY COLLEGE LISTED) 

(ASK ONLY RESPONDENTS WHO ATTENDED A TWO-YEAR COLLEGE) 

(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-d) SHOULD BE ON THE LEFT) 

50. The following is a list of reasons why people might enroll in a two-year college. For each one, 
please tell me if it applies to you. 

1 Yes 

2 No 

X Blank/No Answer 

a. To obtain or maintain skills for a current or future job 

b. To obtain or maintain a license or certification 

c. To take courses before transferring to a four-year college 

d. To obtain a certificate or an Associate’s Degree 
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(ASK ALL) 

(PN: IFQ35=1 INCLUDE VERBIAGE IN PARENS) 

53. Have you attended a vocational, technical, training, or trade program since June 2013? (Do 
not include colleges that you have already mentioned.) 

1 Yes 

2 No 

X Blank/No Answer 
(ASKQ54-Q63IFQ53=1) 

54. Please enter the number of programs you have attended since June 2013? 

(PN: ACCEPT ANSWERS 1-20) 

X Blank/No Answer 

(ASKQ55-Q62IFQ53=1) 

(PN: ASK Q55-Q62 FOR EACH NUMBER OF PROGRAMS SPECIFIED IN Q54) 

(IF Q54=l, INSERT FIRST VERBIAGE; IF Q54>1 OR Q54=8 INSERT SECOND VERBIAGE) 

55. Please enter the name of this program: / Please enter the name of the program you are 
currently attending, or the program you attended most recently: ...Please enter the name of 
the program you attended prior to the last program you mentioned: 


(Specify) 

X Blank/No Answer 

56. Please enter the type of job or career (INSERT PROGRAM) prepares students for: 

(Please enter response below) 

(Specify) 

X Blank/No Answer 

(PN: ALLOW BOTH MONTH AND YEAR) 

57. When did you start attending (INSERT PROGRAM)? Please enter the month and year. 

(PN: ACCEPT 1-12/2013-2014)) 

X Blank/No Answer 

58. Are you still attending this program? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKQ59IFQ58=2) 

(PN: ALLOW BOTH MONTH AND YEAR) 

59. When did you stop attending this program? Please enter the month and year. 

(PN: ACCEPT 1-12/2013-2014)) 

X Blank/No Answer 
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(PN: ALLOW MULTIPLE RESPONSES) 

(ASK Q60 IF 058=2) 

60. Why did you leave (INSERT PROGRAM)? 

Please select all that apply 

1 Completed program 

2 Transferred to a better program or four-year college 

3 Received degree or completed course work 

4 Offered a job 

5 Did not like school or did not get along with other students 

6 Poor grades or failed 

7 Financial difficulties or couldn’t afford to go 

8 Entered military 

9 Moved away from school 

10 Got married 

11 Pregnant or became the father/mother of a baby 

12 Other child care responsibilities 

13 Other family responsibilities 

14 Personal health or substance problems 

15 Other (Please specify) 

X Blank/No Answer 

61. Have you received a certificate, license, or degree from (INSERT PROGRAM)? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKQ62IFQ61=1) 

62. Please enter the kind of certificate, license, or degree: 

(Please enter response below) 

(Specify) 

8 Don’t Know 
X Blank/No Answer 

(ASK Q63. IF Q58=l FOR THE FIRST PROGRAM LISTED IN Q55) 

(PN: INSERT FIRST RESPONSE IN Q55, THE NAME OF THE MOST RECENT PROGRAM) 

(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-g) SHOULD BE ON THE LEFT) 

63. The following is a list of ways that people pay for school. For each one, tell me if it pays for all 
of your expenses, most of your expenses, some of your expenses, or none of your expenses 
attending (INSERT FIRST RESPONSE IN Q55). 

1 All 

2 Most 

3 Some 

4 None 

8 Don’t Know 
X Blank/No Answer 
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a. Parents/spouse/relatives 

b. Work-study program 

c. Grants/scholarships/tuition waivers 

d. Student loans, other loans (e.g., bank) 

e. Employer contribution program 

f. Public assistance (e.g., welfare, unemployment) 

g. Other personal income/savings 


The following are questions about your past and current employment. 

(ASK ALL) 

64. Since you turned 16, have you held a paid or unpaid job, including internships? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKQ64J IFQ64=1) 

64J. Please enter the number of jobs you have you held since turning 16: 

(PN: ACCEPT 1-20) 

8 Don’t Know 
X Blank/No Answer 

(ASKQ64J IFQ64=1) 

(PN: ASK Q65-Q73a AND Q75-Q79 FOR EACH JOB SPECIFIED IN Q64J) 

(IF Q64J=1, INSERT FIRST VERBIAGE; IF Q64J>1 OR Q64J=8 INSERT SECOND VERBIAGE) 

(IF Q64J>1 OR Q64J=8 INSERT PARENS) 

(PN: INSERTTHIRD NOTE IF TREATMENT SAMPLE) 

65. Please enter the name of the company or organization where you work, or worked: / 
Beginning with the most recent job you have held, or your current job, please enter the name 
of the company or organization: ... What is the name of the company or organization you 
worked at prior to the one you just mentioned? 

(If you are currently working at more than one job, please list the one at which you work the most hours or if 
you work equal hours at both Jobs, the one where you have worked the longest) 

(Please include any work study positions) 

(If you participated in the Urban Alliance Internship Program, please list the name of the company or 
organization where you completed your internship.) 

(Company or Organization name) 

X Blank/No Answer 

66. Please enter your position title: 

(Position title) 

8 Don’t Know 
X Blank/No Answer 
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(ASKQ65IFQ64=1) 

67. When did you start working at (INSERT ORGANIZATION)? Please enter the month and year. 

(PN: ACCEPT 1-12/2000-2014) 

8 Don’t Know 

X Blank/No Answer 

(ASKQ65IFQ64=1) 

68. Are you still working there? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKQ69IFQ68=0) 

69. When did you stop working there? Please enter the month and year. 

(PN: ACCEPT 1-12/2000-2014) 

8 Don’t Know 

X Blank/No Answer 

(ASK if Q64=l) 

(PN: IFQ68=1, INSERT FIRST VERBIAGE IN PARENS; IF 068=2, INSERT SECOND VERBIAGE IN 
PARENS) 

70a. (Do/did) you work a different number of hours at (INSERT ORGANIZATION) during the 
school year, than during the summer and other breaks? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKIFQ70a=l) 

(PN: IFQ68=1, INSERT FIRST VERBIAGE IN PARENS; IF 068=2, INSERT SECOND VERBIAGE IN 
PARENS) 

70. About how many hours (do/did) you work at (INSERT ORGANIZATION) per week during the 
school year? 

(Your best estimate is fine) 

(PN: ACCEPT HOURS 0-60) 

8 Don’t Know 

X Blank/No Answer 

(ASKIFQ70a=l) 

(PN: IFQ68=1, INSERT FIRST VERBIAGE IN PARENS; IF 068=2, INSERT SECOND VERBIAGE IN 
PARENS) 

71. About how many hours (do/did) you work there per week during the summer or other breaks? 

(Your best estimate is fine) 

(PN: ACCEPT HOURS 0-60) 

8 Don’t Know 
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X Blank/No Answer 


(ASKIFQ70a=2) 

(PN: IFQ68=1, INSERT FIRST VERBIAGE IN PARENS; IFQ68=2, INSERT SECOND VERBIAGE IN 
PARENS) 

72. About how many hours (do/did) you work at (INSERT ORGANIZATION) per week? 

(Your best estimate is fine) 

1 Hours (0-60) 

X Blank/No Answer 

(PN: IFQ68=1, INSERT FIRST VERBIAGE IN PARENS; IFQ68=2, INSERT SECOND VERBIAGE IN 
PARENS) 

73. How (are/were) you paid at (INSERT ORGANIZATION)? 

(Piease seiect one of the rates iisted beiow) 

1 Hourly 

2 Daily 

3 Weekly 

4 Bi-weekly 

5 Bi-monthly 

6 Monthly 

7 Yearly 

9 The job (is/ was) unpaid 

8 Don’t Know 

X Blank/No Answer 

(ASK IF 073=1-8) 

(PN: IF 068= 1, INSERT FIRST VERBIAGE IN PARENS; IFQ68=2, INSERT SECOND VERBIAGE IN 
PARENS) 

73a. Before taxes or other deductions, what (is/was) your (INSERT RESPONSE FROM Q73) wage 
at (INSERT ORGANIZATION) includingtips and commissions? 

(Piease enter your wages beiow) 

1 Hourly (1-20) 

2 Daily (1-150) 

3 Weekly (1-800) 

4 Bi-weekly (1-1600) 

5 Bi-monthly (1-1750) 

6 Monthly (1-3500) 

7 Yearly (1-60000) 

8 Don’t Know 

X Blank/No Answer 

(ASKQ74IFQ64=1) 

(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-e) SHOULD BE ON THE LEFT) 

74. The following is a list of problems that people might have at work. Thinking about your 
current or most recent job, how often did you have trouble...? 
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1 Never 

2 Only once or a few times 

3 About once a week 

4 Almost everyday 

5 Everyday 

6 Not applicable 

X Blank/No Answer 

a. Getting along with your supervisor 

b. Paying attention while at work 

c. Getting along with your co-workers 

d. Dealing with customers 

e. Arriving on time for work 

(ASKIFQ68=1) 

75. What are your usual duties or activities at this job? For example, filing, selling cars, laying 
brick, customer service. 

(Please enter response below) 

(Specify) 

8 Don’t Know 
X Blank/No Answer 

(ASKIFQ68=1) 

(PN: INSERT VERBIAGE IN PARENS IFQ68=1 FOR MORE THAN ONE JOB) 

76. The following is a list of benefits. Are you eligible for any of the following benefits through 
your employer(s)? By eligible, we mean that the benefit is available to you now, even if you 
have decided to not receive it. 

1 Yes 

2 No 

8 Don’t Know 
X Blank/No Answer 

a. Health insurance 

b. Dental insurance 

C. Paid sick leave 

d. Paid vacation 

e. Employer tuition reimbursement 
(ASKIFQ68=1) 

(PN: INSERT VERBIAGE IN PARENS IFQ68=1 FOR MORE THAN ONE JOB) 

77. How did you find out about your current job, (that is, the one at which you work the most 
hours, or if you work equal hours at both jobs, the one where you have worked the longest)? 

(Please select only one response) 

1 Found on employer’s website or another website 

2 Saw advertisement on campus, in the community, or at a place of business 

3 Recommended by friends or relatives 

4 Recommended by Career Center or at a Job Fair 
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5 I created the position myself or I am self-employed 

6 Other (SPECIFY) 

X Blank/No Answer 

(ASKIFQ68=1) 

(PN: INSERT VERBIAGE IN PARENS IFQ68=1 FOR MORE THAN ONE JOB) 

78. Was there someone who suggested that you apply for your current job(s) or helped you get 
the job(s), other than the person who hired you? Do not include references requested by the 
employer. 

1 Yes 

2 No 

X Blank/No Answer 
(ASKIFQ78=1) 

(PN: INSERT VERBIAGE IN PARENS IFQ68=1 FOR MORE THAN ONE JOB) 

(INSERT RESPONSES 5, 6, 7 IF TREATMENT GROUP) 

79. What was that person’s (or persons’) relationship to you? 

(Please select all that apply) 

1 Parent or Foster Parent 

2 Relative 

3 Friend or Acquaintance 

4 An employer or co-worker (includes current and former) 

5 Urban Alliance Program Coordinator ("PC”) 

6 Urban Alliance Alumni Services Staff 

7 Other Urban Alliance Staff 

8 Caseworker 

9 Teacher 

10 Clergyperson 

11 Other 
99 Refused 

(ASKIFQ68=2) 

80. Have you looked for a job since June 2013? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKIFQ68=2) 

81. What is the main reason you are not working? 

(Please select only one response) 

1 Going to school 

2 Cannot find work 

3 No need or no desire 

4 Taking care of home or family 

5 Previous work was temporary, seasonal, or completed 

6 Laid off 
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7 Quit 

8 Fired 

9 Moved 

10 Former offender or in prison 

11 Temporarily disabled and unable to work 

12 Permanently disabled and unable to work 

13 Changingjobs 

14 Couldn’t afford or find childcare 

15 T ransportation issues or long distance 

16 Not enough skills 

17 Other (SPECIFY) 

X Blank/No Answer 

(ASK ALL) 

(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-h) SHOULD BE ON THE LEFT) 

82. Next, please respond to the following statements. Be honest - there are no right or wrong 
answers! 

For each statement, is that very much like you, mostly like you, somewhat like you, not much 
like you, or not like you at all? 

1 Very much like me 

2 Mostly like me 

3 Somewhat like me 

4 Not much like me 

5 Not like me at all 
X Blank/No Answer 

a. New ideas and projects sometimes distract me from previous ones. 

b. Setbacks don’t discourage me. 

c. I am obsessed with a certain idea or project for a short time but later lose interest 

d. I am a hard worker. 

e. I often set a goal but later choose to pursue a different one. 

f. I have difficulty maintaining my focus on projects that take more than a few months to 
complete. 

g. I finish whatever I begin. 

h. I am diligent. 

This next set of questions asks you about your assets and savings. 

(ASK ALL) 

83. Do you have a checking account? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKIFQ83=1) 

84. What is your approximate current balance in your checking account? 

(1-50000) 

8 Don’t Know 
X Blank/No Answer 
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(ASK ALL) 

85. Do you have a savings account? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKIFQ85=1) 

86. What is your approximate current balance in your savings account? 

(1-50000) 

8 Don’t Know 
X Blank/No Answer 


(ASK ALL) 

87. Do you have any other types of accounts where you have money available to you? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKIFQ87=1) 

(PN: ALLOW FOR MULTIPLE RESPONSES) 

88. What kind of accounts do you have? 

(Specify accounts 1) 

(Specify accounts 2) 

(Specify accounts 3) 

(Specify accounts 4) 

8 Don’t Know 
X Blank/No Answer 

(ASKIFQ87=1) 

(PN: IF R ENTERED MULTIPLE ACCOUNTS IN Q88, INSERT SECOND VERBIAGE IN PARENS) 

89. What is your approximate total current balance in (this / these) accounts? 

(1-50000) 

8 Don’t Know 
X Blank/No Answer 


(ASK ALL) 

90. Do you own any vehicles such as a car, van, truck, jeep, or motorcycle? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKIFQ90=1) 

91. Altogether, how much could you sell these vehicles for? 
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(Your best estimate is fine) 



(1-50000) 

8 Don’t Know 
X Blank/No Answer 


(ASK ALL) 

92. Have you taken out loans to help pay for college or other programs? 

1 Yes 

2 No 

8 Don’t Know 
X Blank/No Answer 

(ASKIFQ92=1) 

93. What is the total dollar amount you have taken out in loans to pay for college or other 
programs? 

( 1 - 100000000 ) 

8 Don’t Know 
X Blank/No Answer 

The following are questions about your family and housing situation. 

(ASK ALL) 

94. Please select your father’s highest level of education: 

(Please select only one response) 

1 Did not complete high school 

2 High school graduate (or equivalent) 

3 Some college (1-4 years, no degree) 

4 Associate’s degree 

5 Bachelor’s degree 

6 Master’s degree or higher 

7 Not applicable 

8 Don’t Know 

X Blank/No Answer 

(ASK ALL) 

95. Please select your mother’s highest level of education: 

(Please select only one response) 

1 Did not complete high school 

2 High school graduate (or equivalent) 

3 Some college (1-4 years, no degree) 

4 Associate’s degree 

5 Bachelor’s degree 

6 Master’s degree or higher 

7 Not applicable 

8 Don’t Know 

X Blank/No Answer 
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(ASK ALL) 

96. Have any of your brothers or sisters gone to college? 

1 Yes 

2 No 

3 Not applicable (Do not have any brothers or sisters) 

8 Don’t Know 

X Blank/No Answer 

(ASK ALL) 

97. Have any other family members that you’re close to gone to college? 

1 Yes 

2 No 

3 Not applicable 

8 Don’t Know 

X Blank/No Answer 

(ASK ALL) 

98. Are there any children who are living with you and in your care? 

“In your care" means that you are legally responsible for the child or have formal custody for the child. 
Informal care arrangements, such as taking care of a sister's child while she is at work, should not be 
Included. 

1 Yes 

2 No 

X Blank/No Answer 
(ASKIFQ98=1) 

99. Please enter the number of children that currently live with you and are in your care: 

( 1 - 10 ) 

X Blank/No Answer 

(ASKIFQ99=1) 

(PN: ASK Q100-Q102 FOR EACH CHILD SPECIFIED IN Q99) 

(IFQ99=1, INSERT FIRST VERBIAGE; IF Q99>1 OR Q99=8 INSERT SECOND VERBIAGE) 

100. Please enter his or her first name: / Please enter the first name of your oldest child: ... Please 
enter the name of your next oldest child: 

(First name) 

X Blank/No Answer 

(ASKIFQ98=1) 

101. What is his or her birthday? 

(PN: ACCEPT 1-12/1-31/2005-2014) 

8 Don’t Know 
X Blank/No Answer 
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(ASKIFQ98=1) 

102. Do you receive child support for this child? 

1 Yes 

2 No 

X Blank/No Answer 

(ASK ALL) 

106. Do you have biological children that do not live with you? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKIFQ106=1) 

107. Are you required by court order to pay child support for these children? 

1 Yes 

2 No 

X Blank/No Answer 

(ASK ALL) 

108. Are you or your partner expecting a child? 

1 Yes 

2 No 

X Blank/No Answer 


(ASK ALL) 

109. What is your current marital status? Are you ... 

(Please select only one) 

1 Single, never married 

2 Living with partner 

3 Married 

4 Separated/divorced/widowed 
X Blank/No Answer 

(NEW 2014) 

(ASK ALL) 

2. Before you turned 18, was there ever a period of four months or more when you did not live 
with at least one of your biological or adoptive parents? 

1 Yes 

2 No 

X Blank/No Answer 

(NEW 2014) 

(ASKIFQ2=1) 

2a. Select the ages below when you did not live with at least one of your biological or adoptive 
parents for four months or more. 
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(Please select all thataoDlv] 


1 

Before your 1st birthday 

2 

1 year old 

3 

2 years old 

4 

3 years old 

5 

4 years old 

6 

5 years old 

7 

6 years old 

8 

7 years old 

9 

8 years old 

10 

9 years old 

11 

10 years old 

12 

11 years old 

13 

12 years old 

14 

13 years old 

15 

14 years old 

16 

15 years old 

17 

16 years old 

18 

17 years old 

19 

18 years old 

X 

Blank/No Answer 

(NEW 2014) 



(ASKIFQ2=1) 

2b. When you were not living with either of your biological or adoptive parents, with whom or 
where did you live? 

(Please select all that apply) 

1 With relatives who were also my foster parents 

2 With relatives who were not my foster parents 

3 With my foster parent(s) who are unrelated to me 

4 With a friend's family (not foster care) 

5 A group home or residential facility 

6 On my own (alone) 

7 Shared housing with a friend or roommate 

8 With my spouse, partner, or boyfriend, or girlfriend 

9 At a homeless shelter or emergency housing 

10 Homeless 

11 College dormitory, fraternity, sorority 

12 Transitional housing 

13 Jail or prison 

14 Job Corps 

15 Other (SPECIFY) 

X Blank/No Answer 

(NEW 2014) 

(ASKIFQ2B=1, 2) 

(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-d) SHOULD BE ON THE LEFT) 

3. Were the relatives...? 
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1 

Yes 


2 

No 


X 

Blank/No Answer 

a. 

Grandparents 

b. 

Aunts or Uncles 

c. 

Older brothers or sisters 


d. Someone else (SPECIFY) 

(ASK ALL) 

110. What best describes your current living situation? Are you ... 

(Please select only one response) 

1 In student housing 

2 In military housing 

3 Living with parents 

4 Living with other adult family member or guardian 

5 Living with significant other 

6 Living with roommates in non-student housing 

7 Living alone 

8 Homeless or living in a shelter 

9 Incarcerated 

10 Other (Specify) 

X Blank/No Answer 

(ASK ALL) 

111. Please enter your current address: 

Your current address does not have to be your permanent address but should be where you currently reside. 
It can be a school address. 

1 (STREET ADDRESS) 

2 (CITY) 

3 (STATE) 

4 (ZIPCODE) 

X Blank/No Answer 

112. Have you been living there since June 2013? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKIFQ112=2) 

(PN: ALLOW BOTH MONTH AND YEAR) 

113. When did you move there? 

(PN: ACCEPT 1-12/2013-2014) 

8 Don’t Know 
X Blank/No Answer 
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(ASKIFQ112=2) 

114. What was the main reason that you moved there? 

(Please select only one) 

1 School (Going to college, leaving college, or wanting an easier commute to school) 

2 Work (Getting a new job, losing a previous job, or wanting an easier commute to work) 

3 Money (Wanting a cheaper place, not having enough money for rent) 

4 Legal problems (Being arrested or incarcerated) 

5 Your health 

6 Wanting to live with someone different 

7 Wanting to be on your own 

8 Needingto help a family member 

9 Wanting to live in a better neighborhood 

10 Needing to find something more permanent 

11 Other reason (SPECIFY) 

X Blank/No Answer 

(ASKIFQ112=2) 

115. Please enter the address were you living before this last move: 

1 (MAILING ADDRESS) 

2 (CITY) 

3 (STATE) 

4 (ZIPCODE) 

X Blank/No Answer 

(ASKIFQ112=2) 

(PN: ALLOW BOTH MONTH AND YEAR) 

116. When did you move there? 

1 (PN: ACCEPT 1-12/1994-2014) 

2 Lived there since birth (did not move there) 

8 Don’t Know 

X Blank/No Answer 

(ASK IF Q116=l AND Q116>6/1/13) 

117. What was the main reason that you moved there? 

(Please select only one) 

1 School (Going to college, leaving college, or wanting an easier commute to school) 

2 Work (Getting a new job, losing a previous job, or wanting an easier commute to work) 

3 Money (Wanting a cheaper place, not having enough money for rent) 

4 Legal problems (Being arrested or incarcerated) 

5 Your health 

6 Wanting to live with someone different 

7 Wanting to be on your own 

8 Needingto help a family member 

9 Wanting to live in a better neighborhood 

10 Needing to find something more permanent 
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12 Lived there since birth (did not move there) 

11 Other reasons (SPECIFY) 

X Blank/No Answer 

(PN: IF Q116>6/1/13, ASK Q115-117 UNTIL Q116<6/1/13, UP TO FIVE TIMES) 

The following questions are about your health. 

(ASK ALL) 

118. In general, would you say your health is... 

1 Excellent 

2 Very good 

3 Good 

4 Fair 

5 Poor 

X Blank/No Answer 

(ASK ALL) 

119. In the past 12 months, have you delayed getting medical or dental care for any reason when 
you really needed it? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKIFQ119=1) 

120. What was the main reason you delayed getting care? 

(Please select only one response) 

1 Didn't know whom to go see 

2 Had no transportation 

3 No one available to go along 

4 Parentorguardian would not go 

5 Didn't want parents to know 

6 Difficult to make appointment 

7 Afraid of what the doctor would say or do 

8 Thought the problem would go away 

9 Couldn't pay 

10 Other (Specify) 

X Blank/No Answer 

(ASK ALL) 

Q121. Do you have health insurance? 

1 Yes 

2 No 

X Blank/No Answer 

(ASKIFQ121=1) 

(PN: INSERT VERBIAGE IN FIRST PARENS AND ITEM 3 IF Q109=3,4) 
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122. What is the source of your health insurance? Would you say it’s through a parent or guardian, 
your college, (your spouse,) the government (like Medicaid), your employer, or something 
else? 

1 Parent/Guardian 

2 College 

3 Spouse 

4 Government (Medicaid) 

5 Employer 

6 Other (SPECIFY) 

8 Don’t Know 

X Blank/No Answer 

(ASKIFQ121=2) 

123. Why are you not covered by health insurance? 

(Please select only one) 

1 Too expensive 

2 No longer eligible 

3 Healthy or don’t need insurance 

4 Too much hassle to stay enrolled 

5 Did not submit paperwork/ documents or pay premiums 

6 Doctors would not accept insurance 

7 Gap in coverage changing plans 

8 Other (SPECIFY) 

X Blank/No Answer 

The next set of questions is about things that have happened to you since June 2013. The following 
questions pertain to only vour personal situation, not that of other family members. These questions 
focus on hardships that many people experience at one time or another. You can choose to skip any 
questions at any time without penalty. 


[ERROR MESSAGE 2] 

(PN: IF A RESPONDENT FAILS TO PROVIDE A RESPONSE TO A CERTAIN QUESTION WITHIN THE 
SERIES Q124-Q146, PLEASE INSERT EM2. THE ERROR MESSAGE SHOULD APPEAR ABOVE THE 
QUESTION MISSED (ON THE SAME SCREEN) IN BOLD BLACK TEXT.) 

EM2 Please remember all your responses will be kept confidential. Your answers are very 
important to us. Can you please take a moment to respond to the question below? 

If you have any questions about the study, please call Kasey Meehan at 484-840-4399. 

(ASK ALL) 

124. Since June 2013, have you received any cash assistance, welfare, or emergency help from a 
state or county welfare program, such as TAN F, fora month or more? 

(Your responses should be based only on your personal situation, not that of other family members) 
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1 Yes 

2 No 



X Blank/No Answer 


(ASK ALL) 

125. Since June 2013, have you received Food Stamps? 

(Your responses should be based only on your personal situation, not that of other family members) 

1 Yes 

2 No 

X Blank/No Answer 

(ASK ALL) 

126. Since June 2013, have you received any governmental housing assistance in paying rent, such 
as through public housing or Section 8? 

(Your responses should be based only on your personal situation, not that of other family members) 

1 Yes 

2 No 

X Blank/No Answer 

(ASK ALL) 

127. Still thinking about things that have happened since June 2013, have you ... had to sleep 
outside or in a shelter on any night? 

1 Yes 

2 No 

X Blank/No Answer 
(AskifQ127=l) 

128. How frequently have you slept in a shelter since June 2013? Would you say... 

1 Once or twice 

2 About once a month 

3 A few times a month 

4 About once a week 

5 A few times a week 

6 Most days 

7 Every day 

X Blank/No Answer 

(ASK ALL) 

129. Still thinking about things that have happened since June 2013, has someone you’re close to 
experienced a major illness or disabling condition? 

1 Yes 

2 No 

X Blank/No Answer 

(ASK ALL) 

130. Still thinking about things that have happened since June 2013, has someone you’re close to 
been incarcerated? 
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1 Yes 

2 No 

X Blank/No Answer 

(ASK ALL) 

131. Still thinking about things that have happened since June 2013, has someone you’re close to 
passed away? 

1 Yes 

2 No 

X Blank/No Answer 

(ASK ALL) 

132. Still thinking about things that have happened since June 2013, have you had to cut the size 
of your meals or skip meals because there wasn't enough money for food? 

1 Yes 

2 No 

X Blank/No Answer 

In answering this next set of questions, think about your experience with illegal or harmful activities. 
Please remember this survey is confidential and your answers will be kept completely private. 

(ASK ALL) 

133. Have you ever been arrested? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKIFQ133=1) 

134. How old were you the first time you were arrested? 

( 10 - 20 ) 

8 Don’t Know 
X Blank/No Answer 

(ASKIFQ133=1) 

135a. Have you been arrested for a felony since June 2013? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKIFQ133=1) 

135b. Have you been arrested for a misdemeanor since June 2013? 

1 Yes 

2 No 

X Blank/No Answer 
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(ASKIFQ133=1) 

136. Have you ever been convicted of a crime as an adult? 

1 Yes 

2 No 

X Blank/No Answer 

(ASK ALL) 

138. Have you ever had at least one drink of alcohol (as in, more than a few sips)? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKIFQ138=1) 

139. During the past 30 days, on how many days did you have at least one drink of alcohol? 

(Your best estimate is fine) 

1 0 days 

2 lor 2 days 

3 3 to 5 days 

4 6 to 9 days 

5 10 to 19 days 

6 20 to 29 days 

7 All 30 days 

8 Don’t Know 

X Blank/No Answer 

(ASKIFQ139=l-7) 

140. During the past 30 days, on how many days did you have 5 or more drinks of alcohol in a row, 
that is, within a couple of hours? 

(Your best estimate is fine) 

1 0 days 

2 1 day 

3 2 days 

4 3 to 5 days 

5 6 to 9 days 

6 10 to 19 days 

7 20 or more days 

8 Don’t Know 

X Blank/No Answer 

(ASKIFQ138=1) 

141. How old were you when you had your first drink of alcohol, other than a few sips? 

( 10 - 20 ) 

8 Don’t Know 
X Blank/No Answer 
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(ASK ALL) 

142. Have you ever used marijuana? 

1 Yes 

2 No 

X Blank/No Answer 
(ASKIFQ142=1) 

143. How many times in the past 30 days did you use marijuana? 
(Your best estimate is fine) 

1 0 times 

2 lor 2 times 

3 3 to 9 times 

4 10 to 19 times 

5 20 to 39 times 

6 40 or more times 
8 Don’t Know 

X Blank/No Answer 

(ASKIFQ142=1) 

144. How old were you when you tried marijuana for the first time? 

( 10 - 20 ) 

8 Don’t Know 
X Blank/No Answer 


(ASK ALL) 

[PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-f) SHOULD BE ON THE LEFT.] 

145. Have you ever... 

1 Yes 

2 No 

8 Don’t Know 
X Blank/No Answer 

a. Used any form of cocaine 

b. Sniffed glue, breathed aerosol, or inhaled paints or sprays to get high 

c. Used heroin 

d. Taken methamphetamines 

e. Taken ecstasy 

f. Taken pills or shots without a doctor’s prescription to get high 
(ASK Q146 FOR EVERY ITEM A-F IN Q145=l) 

[PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-f) SHOULD BE ON THE LEFT.] 

146. In the past 30 days, how many times have you... 

(Your best estimate is fine) 

1 0 times 

2 lor 2 times 
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3 3 to 9 times 

4 10 to 19 times 

5 20 to 39 times 

6 40 or more times 
8 Don’t Know 

X Blank/No Answer 

a. Used any form of cocaine 

b. Sniffed glue, breathed aerosol, or inhaled paints or sprays to get high 

c. Used heroin 

d. Taken methamphetamines 

e. Taken ecstasy 

f. Taken pills or shots without a doctor’s prescription to get high 
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Notes 

1. us Census Bureau, 2012 American Community Survey, one-year estimates. 

2. Estimates are for the graduating class of 2013. Baltimore estimates are from 2013 Maryland Report Card, 
"Baltimore City Graduation Rate: Four-Year Adjusted Cohort,” Maryland State Department of Education, 
accessed May 20, 2014, 

http://www.mdreportcard.org/CohortGradRate.aspx?PV=160:12:30:XXXX:l:N:0:13:l:l:0:l:l:l:3. DC 
estimates are from “DC 2013 Adjusted Cohort Graduation Rate,” District of Columbia Office of the State 
Superintendent of Education, accessed May 20, 2014, 

http://osse.dc.gov/sites/default/files/dc/sites/osse/publication/attachments/DC%202013%20ADJUSTED%2 
0COHORT%20GRADUATION%20RATE%20state%20summary_0.pdf. The DC percentage does not include 
DC public charter schools. 

3. Current Population Survey, “School Enrollment,” table 1, Enrollment Status of the Population 3 Years Old and 
Over, by Sex, Age, Race, Hispanic Origin, Foreign Born, and Foreign-Born Parentage: October 2013. 

4. National Center for Education Statistics, 2014 Digest of Education Statistics, table 302.30, Percentage of Recent 
High School Completers Enrolled in 2-Year and 4-Year Colleges, by Income Level: 1975 through 2013. 

5. National Center for Education Statistics, 2014 Digest of Education Statistics, table 326.10, Graduation Rates of 
First-Time, Full-Time Bachelor's Degree-Seeking Students at 4-Year Postsecondary Institutions, by 
Race/Ethnicity, Time to Completion, Sex, and Control of Institution: Selected Cohort Entry Years, 1996 
through 2006. 

6. 2009-13 American Community Survey 5-year estimates, "Median Earnings in the Past 12 Months (in 2013 
Inflation-Adjusted Dollars) by Sex by Educational Attainment for the Population 25 Years and Over.” 

7. College Board, 2009, "Lifetime Earnings by Education Level,” figure 1.2, Expected Lifetime Earnings Relative to 
High School Graduates, by Education Level. 

8. Current Population Survey Household Data 2014, table 3, Employment Status of the Civilian Noninstitutional 
Population by Age, Sex, and Race; and table 4, Employment Status of the Hispanic or Latino Population by Age 
and Sex. 

9. The GED consists of four tests that certify passers' high school-level academic skills. 

10. In July 2014, the DC minimum wage rose to $9.50 per hour, and in July 2015 it reached $10.50 per hour. These 
adjustments occurred after the participants in this study had completed their internships. 

11. For Baltimore, we used the “2011 HSA English Data” and “2011 HSA Algebra Data” data files from "Data 
Downloads,” 2013 Maryland State Report Card, accessed June 17, 2013, 

http://msp.msde.state.md. us/downloadindex.aspx?K=99AAAA. For DC, we used data reports for each school 
from 2011, accessed through “Assessment and Accountability in the District of Columbia,” District of 
Columbia Office of the State Superintendent of Education, accessed on June 17, 2013, 
http://nclb.osse.dc.gov/reportcards.asp. 

12. National Center for Education Statistics, "Public Elementary/Secondary School Universe Survey Data, 2009- 
10,” accessed June 6, 2013, http://nces.ed.gov/ccd/pubschuniv.asp. 

13. The random effects model modifies the earlier regression framework to 
ysi* = piindsi + p2neighsi -r as + esi 

such that as is the high school specific effect on ysi. In a random effects model, the assumption is that the high 
schools included in the analysis are a subset of a greater pool of high schools, such that as is distributed 
normally with mean zero and variance oa2.This assumption allows the regression to use both between and 
within variation in the data. 

14. In this report, African American refers to non-Hispanic African American, white to non-Hispanic white, and other 
to non-Hispanic other. 
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15. There are exceptions; some programs appear to be targeted toward (or more attractive to) males. For example, 
the National Guard Youth ChalleNGe Program is 88 percent male (Millenky et al. 2011), and Job Corps is 59 
percent male (Schochet et al. 2006). 

16. In recent years, many Baltimore and Washington, DC, schools have become certified to offer free lunch to all 
students. Therefore, eligibility of individual students is no longer determined; the statistics shown here give an 
estimate of what eligibility would be in these schools if it were still determined at the student level. In the 
2012-13 school year, students from a four-member household with income below $42,643 qualified for a 
reduced-price meal, according to federal guidelines. 

17. Of the 965 youth for whom data are available on GPA as of junior year, school records provided information 
for 657 youth (68 percent). For, 253 youth (26 percent), this information was provided by a school counselor. 
Finally, 55 youth (6 percent) provided this information directly. 

18. We define completion as remaining involved in the internship and workshops as of June 1; a portion of 
participants were unable to continue through the summer because of other commitments. 

19. There was also a modest amount of program attrition among those who completed the pre-work training but 
did not begin an internship. 

20. We were unable to determine senior year GPA, and instead had to rely on cumulative GPA, because some 
schools only reported cumulative GPA. 
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